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ABSTRACT 



This proceedings document is comprised of the 17 papers, 
panel presentations, and work shares presented at a 1998 conference on 
institutional research. The papers are: (1) "Description of the UDAES 

Project: A Study of Undergraduate Academic Experiences" (Karen W. Bauer); (2) 
"Transforming Your Campus: Mixed Methodology in Institutional Research" (R.J. 
Brodnick and Tracy L. Pol insky) ; (3) "Factors Affecting Grading Practices" 

(David X. Cheng) ; (4) "Designing Graduate Admission Studies To Influence 

Campus Transformation" (Anne Marie Delaney) ; (5) "Using Microsoft Access to 

Facilitate the Sampling and Distribution of an Alumni Survey" (Karen DeMonte 
and Karen W. Bauer) ; (6) "The Relationship Between Student Success in College 

and Assessment for Remedial Assistance" (Michael J. Keller and Monica 
Williams -Randall) ; (7) "A Formula for Writing Environmental Scans for 

Community Colleges" (Marcia M. Lee) ; (8) "Financing Education: An Analysis 

from the Results of the Survey of Graduates 1989-1997" (Qing Lin Mack and 
Jennifer A. Brown) ; (9) "The Impact of Outsourcing Auxiliary Enterprise on 

Educational Expenditures Per FTE" (Joan B. McDonald and Barbara R. Sadowski) ; 
(10) "Math Achievement and Math Course Taking" (Denise Nadasen) ; (11) "Using 

Cluster Analysis for Transcript Analysis of Course-Taking Patterns" (Wayne S. 
Obetz) ; (12) "From Database to Data Warehouse" (Wayne Obetz and Jane 

Grosset) ; (13) "They'll Surf but They Won't Swim: Student Reluctance to Apply 

to College Online and Implications for Web-Based Survey Research" (Sarah 
Parrott and Jennifer McKnight) ; (14) "On-Line News vs. Traditional Media: 

Student Preference Regarding the Acquisition of Current Events" (Dawn 
Geronimo Terkla and Jennifer McKnight) ; (15) "Using Predictive Modeling to 

Target Student Recruitment: Theory and Practice" (Emily Thomas , Gayle 
Reznik, and William Dawes) ; (16) "Comparing Administrative Satisfaction in 

Public and Private Higher Education" (James Fredericks Volcan and Kelly 
Parley) ; and (17) "Factors Affecting Academic Outcomes of Underprepared 
Community College Students" (Jishen Zhao) . Also included are the conference 
program and the 1998 membership list. (Some papers contain references.) (DB) 
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Dear NEAIR Friends and Colleagues: 

The 25*^ Anniversary Conference of the North East Association for Institutional Research was celebrated at 
the Sheraton Society Hill hotel in the historic district of Philadelphia on November 14-17, 1998. While 
these Proceedings include much of the intellectual content shared at the meeting, they cannot capture the 
spirit of camaraderie, mutual respect and support, professional commitment, and outrageous fun that 
permeated the conference. 

It is NEAIR’s tradition to entrust conference planning and execution to the program chair and the local 
arrangements chair. Once again, the association’s faith in this approach was superbly rewarded. Helen 
Schneider organized the largest program in NEAIR history, with 14 pre-conference workshops, three 
general sessions, and over 60 concurrent sessions. Her program theme, “The Future is Now: The Role of 
Institutional Research in Campus Transformation” was carried forward throughout from Bill Flynn’s 
opening keynote, through Dave Hollowell’s general session on Monday, to the closing panel of NEAIR 
past presidents (Lay, Middaugh, Pagano, Terenzini, and Terkla). Helen’s program attracted a record 
number of presenters, which required some innovative scheduling arrangements, and a near-record 253 
conference attendees. 

Local arrangements chair Steve Thorpe was equally superb in hosting the anniversary conference. The 
work of putting on a four-day conference for over 250 people is daunting, and requires a person who is 
organized, persistent, inventive, and highly skilled in negotiations, team building, interpersonal skills, and 
follow-through. Steve worked with the staff at the Sheraton Society Hill to deliver a conference that ran 
smoothly and met all the needs of the attendees. Though my debt to Steve was fully paid when the photos 
of me in full Mummer’s gear were widely circulated, I nevertheless will thank him once again here for all 
his efforts and look forward to future SIG meetings together. 

Brenda Bretz, our extraordinary membership secretary, once again handled all conference registration 
processes with her usual competence and good humor. The only sad note for NEAIR was that Brenda 
announced her retirement from this position. So Philadelphia was Brenda’s final conference in this role. I 
join all of NEAIR in thanking Brenda for years of effort in keeping the Steering Committee and conference 
planners on schedule. 

It was my great pleasure to present Fred Volkwein the association’s first Distinguished Service Award. 
Fred’s contributions as a scholar, practitioner, and mentor have been of immense value to NEAIR and the 
profession of institutional research. I was also very pleased that former NEAIR president Pat Terenzini 
was present to see Fred receive his award. As Fred was first to acknowledge in receiving his award, Pat 
blazed many of the paths Fred so fruitfully followed. 

I want to also thank Pat for allowing me to reproduce his seminal article on the nature of institutional 
research in our 25*^ anniversary publication. Celebrating the past. Preparing for the future. Pat’s article, 
along with those contributed by Jack Dunn, Fred Volkwein, and Frank Schmidtlein, provided an invaluable 
primer in IR for newcomers and experienced practitioners alike. Adding in Christine Scarince’s history of 
NEAIR article and the personal reflections essays by 13 charter members and past presidents. Celebrating 
the past. Preparing for the future — beautifully produced by Pat Diehl — became the historical NEAIR 
keepsake I had hoped for when the idea was first conceived. 

Finally, a special thank you to Corby Coperthwaite for agreeing to produce these Proceedings for a third 
consecutive year. Her work in collecting, editing, and producing this document, in both electronic and 
paper form, is greatly appreciated. Her recent election to the Steering Committee should free her of this 
task in the future, but NEAIR will continue to benefit from Corby’s good ideas and dedicated efforts. 

Craig A. Clagett 
NEAIR President, 1997-98 
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Description of The UDAES Project: A Study of Undergraduate Academic Experiences 



Karen W. Bauer, Ph.D. 

Assistant Director, Institutional Research & Planning 
The University of Delaware 



Abstract 

Many IR officials are involved in assessing the academic transformations that occur on 
our campuses. This presentation focuses primarily on the research design from the 
research project, University of Delaware’s Academic Experiences Study (UDAES). 
Funded through a National Science Foundation Award, this project documents the 
effectiveness of the University’s Undergraduate Research Program and evaluates its 
educational affects on students and faculty involved in this program. Data gathered from 
265 students in May, 1997 will serve as baseline measures for growth and change over 
the next three years. 



Introduction 

Recent calls for evaluation of higher education, especially doctoral-granting 
research universities (Boyer, 1997), have prompted institutional researchers to become 
more involved in studies examining and documenting the cognitive and social gains for 
college students. In addition to documenting levels of satisfaction and use of campus 
facilities, researchers are contributing to our understanding of how college effects 
students. Astin (1993), Chickering (1969), and Pascarella and Terenzini (1991) have 
offered valuable information on characteristics, needs, and developmental changes of 
baccalaureate students. Terenzini, Pascarella, and Blimling (1996) added to this 
information by arguing that much learning occurs outside the classroom and that 
academic and social integration are crucial factors in determining a student’s success in 
college. 

Astin (1993) and Pace (1984) believe that students leam best when they invest 
their physical and psychological energy in college activities. Involvement in 
extracurricular and cocurricular activities is both qualitative and quantitative and when 
students put forth greater involvement, they reap greater benefits. Students who become 
highly involved in campus activities will, therefore, acquire the highest level of new 
cognitive and social skills. 

Description of The RAIRE Award and UD Research Design 

Funded through a National Science Foundation Award that began in February, 
1997, the UDAES Project (University of Delaware Academic Experiences Study) is a 
major component of the RAIRE Award (Recognition Award for the Integration of 
Research in Educaiton). This award recognizes the institution’s success at integrating 
research in undergraduate education. The award will enable institution officials to 
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engage in a multi-faceted four-year study to document the effectiveness of the 
University’s Undergraduate Research Program (URP) and examine the benefits to faculty 
and students who become involved in undergraduate research. The major portion of this 
presentation will be a discussion of the components of this research project. The UD 
RAIRE Award includes four major components: 

I. Summative analysis of student evaluations available at Undereraduate 
Research Program Office . A sample of approximately 200 evaluations completed since 
1982 by Science and Engineering Scholars were examined and analyzed to indicate level 
of satisfaction, self-reported gains, and the perceived difficulties and drawbacks from 
students and faculty who have been involved with the URP. Sixteen categories of 
responses were established in this content analysis, including increased technical skills, 
working with others, increased desire to learn, and satisfaction with stipend. Overall, the 
Science and Engineering Scholars reported high satisfaction and positive academic 
progress as a result of their undergraduate research experience. Ninety-six percent of the 
respondents reported increased technical skills, 57% said that their research experience 
helped them learn to act independently, and 45% said the experience was helpful in terms 
of their future career and/or giving them insight into the graduate schools experience. In 
addition, only 4% said they did not receive enough attention from faculty mentors, and 
14% reported they felt the research stipend was not adequate. 

II. Analysis of Impact for Faculty and Graduate Students . Through focus group 
interviews and/or a questionnaire, data will be gathered from faculty and graduate 
students who supervise undergraduate students in laboratories to examine their 
satisfaction and challenges faced from undergraduate involvement in research. This 
portion of the research project will occur in the 1998-99 year. 

III. For Alumni Who Were Involved with URP as a student . A UD Alumni 
Survey was sent to a sample of approximately 2,300 UD alumni, half of whom were URP 
students. We hypothesized that alumni who engaged in undergraduate research would 
report higher satisfaction, greater academic and personal gains, and report that their 
undergraduate experiences were more influential for graduate school and/or career, 
compared to those students who did not engage in undergraduate research. Alumni 
satisfaction with the University and the URP, self-perceptions on acquisition of research 
and other skills through the URP experience will be documented. 

TV. Analysis of Impact for Students . For current students, a sample of 
undergraduate students was identified in Spring, 1997 to participate in the four-year 
UDAES study. Student demographic variables including gender, ethnic, SAT, predicted 
grade index, age, and number of credit hours attempted divided by the number hours 
completed will be gathered from the university student records system. Each year, the 
cohort of students will complete a battery of instruments to monitor level of critical and 
creative thinking, college satisfaction, and level of campus involvement. Because no one 
or two instruments were able to capture the information we hoped to gather, we will 
create an academic profile of students with data from the following instruments: 
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• Watson Glaser Critical Thinking Appraisal (WGCTA) 

• Consequences Test (CQ) 

• NEO Five Factor Inventory (NEO-FFI) 

• College Student Experiences Questionnaire (CSEQ) 

• Reasoning About Current Issues (RCIT) 

The Watson Glaser Test of Critical Thinking (WGCTA) is a paper-pencil 
measure that includes five subtests that tap different aspects of the critical thinking 
domain. The five areas are: the ability to make inferences; recognize unstated 
assumptions or presuppositions; deduce whether certain conclusions necessarily follow 
from information in given statements; interpret information by weighing evidence and 
deciding if generalizations or conclusions are warranted; and evaluate different 
perspectives and distinguish between strong and weak arguments. 

Group norms are available, created from 4,571 adults. The WGCTA reports 
internal consistency via Cronbach’s alpha coefficient r = .81, and test-retest reliability r 
= .81 (p <.001). Undergraduate students with higher GPAs were found to score better on 
the WGCTA (Steward & Al-Abdulla, 1989) and relationships have been found between 
WGCTA and grades in college chemistry (Hurov, 1987), physics (Wilson & Wagner, 
1981), as well as GPA, degree attainment, and employment in related field (Hildebrandt 
& Lucas, 1980). 

The Consequences Test (CQ; Guilford & Guilford, 1980) measures the creativity 
with which one thinks about hypothetical situations. When asked to hypothesize ‘what 
would happen if...’, one is asked to list (in a short time period) alternative outcome 
scenarios for five fictitious problems or situations. Answers to these scenarios indicate 
the creativity with which one thinks about problems. Answers are either obvious which 
indicates a direct and immediate result, and displays less awareness of social, economic, 
and cultural ramifications, or remote which indicates a change that is more distant, either 
temporally or geographically, or shows a substitute way of adjusting to the changed 
situation. Creativity is indicated by the combination of greatest number of acceptable 
responses with largest proportion in the “remote” category, i.e., largest number of 
solutions that are (1) original and (2) relevant to the problem. 

The CQ boasts high stability measures. Split-half reliability was .72 for males; 

.83 for females (Ibrahim, 1976). Pre- and post-test correlations were found to be .78 
(Frederiksen & Evans, 1974) and .69 (Hilton, 1968), and construct validity as evidenced 
through factor loadings ranged from .41 to .71 for the obvious score and .31 to .70 for the 
remote score. 

Many researchers believe there is a relationship between creative thinking and 
high intelligence. Goor (1975) found a combination of CQ remotes scores and ‘clever’ 
responses to Plot Titles correlated .35 with SAT scores, and Gundlach and Gesell (1989) 
found significant correlations (r=.22) between remote and obvious responses and 
nonverbal IQ. In addition, Wade (1971) used CQ in combination with other creativity 
tests and found a correlation of .36 (p < .001) with verbal intelligence, and Gordon (1972) 
found that students who qualified for an honors program were generally more creative as 
measured by CQ scores and other tests of divergent production. 

The NEO Five Factor Inventory (NEO-FFI; Costa & McCrae, 1987) measures 
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five broad domains, or dimensions, of personality and offers insight into how one solves 
problems and interacts with others. The five broad domains are: 

• Neuroticism - level of adjustment and emotional stability 

• Extraversion - level of sociability and consequent behaviors that occur as a result of 
interactions with others 

• Openness - active imagination, aesthetic sensitivity, attentiveness to inner feelings, 
preference for variety, intellectual curiosity, and independence of judgement 

• Agreeableness - level of sympathy and altruism toward others, eagerness to help 

• Conscientiousness - ability to manage impulses and desires and the process of 
planning, organizing, and carrying out tasks 

Internal consistency for the five NEO scales was calculated with a sample of 1,539 
subjects. Coefficients were .86, .77, .73, .68, and .81 for N, E, O, A, and C, respectively. 
Convergent correlations (to demonstrate validity to full scales) ranged from .56 to .62. 
The FFl scale accounted for approximately 85% as much variance in convergent criteria 
as do the full factor scores. Research indicates that openness is related to intelligence and 
divergent thinking (McCrae, 1987). Conscientious students are well-organized, 
purposeful, and persistence and these traits lead to higher academic achievement 
(Digman &Takemoto-Chock, 1981). 

The College Student Experiences Questionnaire (CSEQ; Pace, 1984) examines 
students’ self-report quality of effort they put forth with various college activities. In 
addition, it examines students' satisfaction with the campus environment, perceptions of 
the campus environment (emphasis on scholarly, aesthetic, and vocational issues) and 
perceived annual gain in a series of academic and personal items. 

Cronbach reliability coefficients for the CSEQ ranged from .81 to .91, and 
Guttman-scale analysis and factor analysis for content coherence were very high (Kuh, 
Vesper, Connolly, & Pace, 1997). Revised (3rd edition) norms were based on 15, 133 
students at 18 Research Universities; 50,188 students at 66 total institutions. 

Pike (1995) found that student reports of experiences as measured on the CSEQ 
were highly correlated with achievement test scores. In addition, Kuh, Vesper, Connolly 
and Pace (1997) found that self-reported gains are greatest for questions related to their 
discipline or major. Several researchers, including Astin (1993), Ballou, Reavill, & 
Schultz (1995); Bauer (1992; 1997) and Pace (1981; 1986), found that students make 
larger academic gains when they are involved in campus activities. 

The Activities and Interests Inventory was created to document the kinds of 
extra-curricular activities, level of involvement, and amount of time students report 
spending on each during the year. It offers additional information on specific activities 
(i.e., sports, clubs, community volunteer activities, and employment) and determines 
whether URP students have the same experience with extracurricular activities as non- 
URP students. 

The Reasoning About Current Issues Test (RCIT; Wood, 1997) is designed to 
measure one's level of epistemological thinking and is based on King and Kitchener's 
(1994) Theory of Reflective Judgement. The RCIT is a paper-pencil survey that examines 
students’ beliefs about the nature of knowledge and learning and their personal 
epistemology. Items are categorized into five more or less independent dimensions. 
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beliefs about the structure of knowledge, certainty of knowledge, source of knowledge, 
control of knowledge acquisition, and speed of knowledge acquisition. 

Survey Administration 

In the Spring, 1997, a letter was sent to each new freshman student in select 
majors asking them to join this study. Of the approximately 500 letters sent to freshmen, 
265 students met with me during March, 1997 to complete the set of profile instruments. 
Four meetings times were designated and most of the 265 students met with me during 
this time. For those who could not attend the group meeting, I scheduled individual 
meetings to have the student complete the set of instmments. Table 1 shows the 
breakdown of instruments completed each year. As shown, due to the stability of the 
NEO outcome, it will be administered only once (during the freshman year). The 
WGCTA will be administered in years one and four only. The CSEQ, CQ, and Activities 
& Interests Inventory will be administered each year, freshman through senior years. 

Hypotheses for UDAES Study 

We hypothesize that the research experience will initiate or support students’ 
active participation in the learning process. We hypothesize that students who are 
involved with URP will have higher persistence rates, earn higher grades, sharpen their 
ability to synthesize information, think more critically and creatively, become more 
sophisticated about the problem-solving process, and frame issues in a broader context. 

In addition, greater participation in learning and an understanding of the nature of 
knowledge will allow the UR student’s personal epistemology to become more complex. 
Specifically, our hypotheses for the UDAES study are that students who engage in 
undergraduate research will: 

• have higher persistence and graduation rates; 

• select more challenging courses of study; 

• develop a greater ability to synthesize information; 

• think more critically and more creatively; 

• become more sophisticated in problem-solving; 

• develop greater tolerance for ambiguity; 

• frame issues in a broader context; 

• develop greater investment in their academic growth; and 

• develop greater attachment to the University community. 

Data Collection 

Data for the students in undergraduate research will be compared with a matched 
sample of students who do not engage in undergraduate research. A total of 265 second 
semester freshmen completed Year One instruments in May, 1997. This cohort of 
students will be followed throughout their baccalaureate work at UD and asked to 
complete select measures at the end of each academic year. It is hoped that 
approximately one-third of this cohort will become involved in URP, beginning in 
summer or fall, 1998. Data collected will enable me to chart annual progress and 
differences between URP and non-URP students. 
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Summary of Year One Findings 

Data was obtained from 265 freshmen in May 1997. Fifty-eight percent were 
women, 77% were white, 35 % honors students, and were majoring in Chemical, Civil, 
Electrical, and Mechanical Engineering; Computer Science; Chemistry; Biology; Animal 
Science; Psychology. The mean SAT score for the group was 1,202, and their average 
cumulative GPA for Spring 1997 was 3.02. A preliminary cluster analysis revealed three 
distinct levels of effort or involvement for UDAES students based on CSEQ scores (see 
Figure 1). Data from the completed UDAES students in May, 1997 will serve as 
baseline measures for future academic growth and change over the baccalaureate 
experience. We will chart change from year to year as well as freshman to senior year 
change. 

Table 1 

Instruments Used to Create Academic Profile 



Instrument: 


Measures: 


Administered When: 




College Student Experiences 
Questionnaire (CSEQ) 


level of involvement and quality of 
effort in campus activities 


each spring 


Consequences Test 
(CQ) 


the creativity with which one 
thinks about hypothetical problems 


each spring 


Activities & Interests 
Inventory 


the kinds and frequency of student 
involvement in extracurricular activ., 
employment, research activities 


each spring 


Watson Glaser Critical 
Thinking Appraisal 
(WGCTA) 


five aspects of critical thinking - freshman and 
inference, recognition of assumptions, senior years 
deduction, interpretation, & evaluation 
of arguments 


NEO Five Factor 
Inventory (NEO-FFI) 


five domains of personality. Offers 
insight into how one solves problems 
and interacts with others. 


freshman year 
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Figure 1 

Exploratory Cluster Analysis, Year One CSEQ, WG, and NEO-PTI 
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Abstract 



Researchers play crucial roles in campus transformation via information. This 
information must best represent their institution’s reality. By combining data collection 
methods, researchers can help to isolate leverage points for change. This paper reviews 
multiple methods, discusses data integration, and gives an example where mixed 
methodological research induced successful transformation. 
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Transforming Your Campus: Mixed Methodology in Institutional Research 

Institutional researchers can play a critical role in transforming their campuses by 
providing information that can serve as the foundation for institutional change. It is 
important, however, that the information they provide best represents the reality of their 
institution. When given a request for data regarding a current problem, too frequently 
researchers respond with the "best numbers" presently available in institutional databases. 
This practice results in reports that are related to the problem but do not quite meet the 
needs of decision-makers or answer the original question. Certain issues require a more 
careful approach where researchers should contemplate all available research methods 
and data before choosing those that will yield the most accurate and pertinent 
information. 

There are two issues that specifically impact the roles of institutional researchers 
in campus transformation. First, higher educational institutions serve a special role in 
their environments. In higher education, transformation is a process of change that 
specifically focuses on individuals. As opposed to a factory, where changes in assembly 
lines or chemical processes can lead to substantial transformations in products, 
educational institutions provide services that depend on interactions among and within 
human beings. Transformation in this environment depends heavily on the evolution of 
the mindsets and realities of those that provide the education and those that render the 
support services that complement learning. 

The second issue that impacts the roles of institutional researchers relates to what 
is perceived to be real in their institutions. Often, there is an expectation of a single 
organizational reality. Researchers tend to investigate stable, unidimensional traits in 
each institution and assume that all employees perceive the campus in the same way. A 
more accurate description is that there are always multiple institutional realities and that 
each individual, whether on or off campus, has a unique perception of the institution. A 
portion of this perception may be shared and groups may have considerable collective 
perceptions. Understanding the differences and commonalties in perceptions of the 
campus can help guide more advanced research and transformation strategies. Research 
strategies then, should address these multiple institutional realities. 

What then, given these two issues, is the role of institutional research? Our 
suggestion is that institutional researchers focus on creating a shared institutional reality 
by educating and involving institutional members (faculty, staff, administration, and 
students) in dialogue regarding issues leading to campus transformation. The strategy 
discussed in this paper is to provide more complete information through mixed 
methodology. 



What is mixed methodology? 

Mixed methodology research utilizes multiple methods (typically both 
quantitative and qualitative) in an attempt to obtain the most complete picture possible of 
the phenomena of interest. This approach provides researchers and decision-makers at an 
institution with a more comprehensive, multi-perspective understanding of the issue 
under investigation than any one single method alone could provide. The complexity of 
many educational institutions’ decisions can be more fully addressed by combining 
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quantitative methods’ ability to produce a panoramic view with qualitative methods’ 
capability of obtaining an in-depth description. By integrating multiple data collection 
methods, researchers can more accurately and thoroughly depict their college or 
university and the issues that surround the problems at hand. The next few paragraphs 
explain what data sets are and give an overview of how methods can be mixed. 

While many investigations rely on a single data set to provide an answer to a 
research question, mixed methodological inquiry involves the use of multiple data sets. 
Multiple data sets emerge when various data collection strategies are employed. Two sets 
of data retrieved from the same method, then, could not be considered multiple data sets. 
In other words, a researcher who uses data obtained from two different administrations of 
the same survey instrument is not applying multiple data sets. One set of data obtained 
from the survey and another procured via a database extraction, however, would 
constitute multiple data sets. 

In mixed methodology research, as in all research, the first step must be to 
identify and define the research question. Once the research problem has been 
established, the next step is to decide what type of methodology is most fitting to the 
situation. Given the variety of designs, certain methods are more appropriate than others. 
When the research question is of the "what," "who," "when," or "where" variety, 
quantitative measures are usually the method of choice. For instance, for questions such 
as where incoming freshmen come from geographically, or what percentage of biology 
majors have a 3.0 GPA or higher, an extraction from existing databases and descriptive 
statistics should suffice. If, on the other hand, the research question is of the "why" or 
"how" nature, qualitative or more advanced quantitative methods are typically 
appropriate. If, for example, the question surrounds issues of how students might react to 
a new campus policy, qualitative research techniques such as interviews or focus groups 
would prove useful. 

In today’s complex higher education environments, the research problem is often 
of a nature that warrants mixed methodology. Because of the time one must invest in 
order to conduct mixed methodological research, the researcher must determine, quite 
frankly, if the problem is worth the attention. If it is a complex yet critical issue, then 
mixed methodology is probably in order. 

Perhaps the most important advantage of conducting mixed methodological 
research is having the ability to investigate phenomena from various vantage points and 
in different ways. Mixed methodological research, then, can be likened to taking a 
journey; the descriptions of this trip by an individual who took an airplane to reach a 
destination will be very different from those of someone who walked to that same 
destination. The view from the airplane allows the traveler to see quick glimpses of a lot 
of things in a short period of time. He or she may be able to recount, for instance, within 
one hour seeing five bodies of water, but may not be able to describe much beyond that. 
While the experience of the walker is much slower, thus preventing the observance of 
many things at once, those things that the walker does observe are observed in greater 
contextual and personal detail. Mixed methodological research is like asking many who 
have traveled the route to describe what they saw. It should be clear that when all of the 
various descriptions are combined, a very comprehensive picture is drawn. One person’s 
account is not more or less correct than another person’s, just very different. Quantitative 
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and qualitative research methods are not competitors, then, but complements of each 
other. Each type of research, when used well, has the ability to uncover facts or 
perceptions that the other method, due to its very nature, would probably have not 
unearthed. 



A Review of Quantitative Methods 

Quantitative methods such as descriptive and inferential analyses adhere to 
stringent procedures that include sampling, manipulation, and control. Consequently, the 
results obtained via quantitative means are generalizable to a larger population. The 
strength of quantitative measures lies in their standardized and numerical results that are 
appropriate for statistical analysis. The data collected can identify relationships among 
variables and depict the characteristics of populations. Quantitative research methods are 
a fundamental component of social research today. At times, the information needed by 
an institutional researcher, such as enrollment figures and demographic information 
regarding students at the institution, is already available in a campus-wide database. 

While other quantitative techniques are important, most of the day-to-day investigations 
of the institutional researcher are composed of the statistical manipulations of data 
extracted from databases. 

A commonly used technique in quantitative research endeavors is the survey. 
Because many of the instruments employed in survey research are standardized, the 
results can be generalized to a population beyond those that were surveyed. By utilizing 
standardized instrumentation, the researcher can acquire a great deal of information 
(about a large number of people) relatively quickly and easily. Standardization is not only 
a prerequisite to achieving reliable results, but allows researchers to compare and contrast 
results across groups of individuals who responded to the survey instrument. 

Experimental research, perhaps because it requires manipulation and control, is 
not typically used in institutional research. Institutional researchers, instead, tend to focus 
on phenomena as they already exist. For this reason, quasi-experiments are more likely to 
be used in institutional research settings. In quasi-experimental research, the investigator 
looks at differences between two populations that are as similar as possible in all 
characteristics except for the one of interest. The researcher, then, is able to attribute, 
with some level of certainty, differences in the dependent variable between the two 
groups to the one salient pre-existing difference in their characteristics. 

Similarly, an investigator may choose to acquire data via direct observation. 

Direct observation, unlike survey research that requires participants to state their 
opinions, requires the researcher to record the behaviors of other individuals. The 
researcher can then develop a data set based on the frequency of these occurrences. 

Analyzing quantitative data sets involves knowing when and how to apply the 
appropriate statistical procedures. When description is in order, the researcher must 
calculate, generally with the help of statistical computer software, frequencies, 
distributions, measures of central tendency, graphs, and tables. These descriptive 
statistics provide a good first look at quantitative data. When the researcher aims to 
discover if and to what extent variables in the data set are related to each other, or co- 
vary, a correlation is the statistic to be used. Correlations, conducted easily and quickly 
with programs such ^ SPSS, tell the researcher the extent to which change in one of the 
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variables of interest is associated with change in the other variable. 

When the researcher is interested in comparison between or among groups, 
inferential statistics are required. The researcher conducts T-tests when looking at the 
differences between two groups and ANOVA when interested in the differences among 
more than two groups. These statistics, in sum, provide the researcher with a number that 
states whether or not the differences in the dependent variable between or among the 
groups can be attributed to the independent variable and with what level of certainty the 
researcher may accept these results. These comparative statistics, while conunonplace in 
certain areas of applied psychology, are utilized less frequently in institutional research, 
because they tend to be applicable to experimental or quasi-experimental data. 

Regression analyses are appropriate when prediction is the desired outcome. 
Linear regression analysis, for instance, allows the researcher to ascertain in what way a 
change in one variable is associated with change in the other. Namely, if the investigator 
knows the relationship between two variables (represented by an equation), he or she can 
predict the value of one variable if the value of the other is known. An institutional 
researcher who is interested in the relationship between SAT scores and college GPA, for 
instance, can develop a linear regression model involving these two variables. Then, if a 
student’s SAT score is known, the researcher can insert it into the equation, and 
determine what the student’s college GPA will be. Other types of regression operate by 
the same principle (prediction of one variable based on the other), but involve different 
statistical procedures. Through logistic regression, for example, the researcher attempts to 
find what variables, when taken together, are the best predictors of a single categorical 
variable. Other statistical procedures, occasionally used in institutional research, include 
multivariate statistics, factor analysis, and structural equation modeling. 

Regardless of the statistical procedure used, the quantitative research process can 
be compared to viewing a painting in an art museum. Upon looking at the picture, the 
quantitative researcher would most likely step up close to the painting to scrutinize the 
individual dots of paint that comprise the picture. He or she might look closely at the 
characteristics of the artwork, counting how many dots the painting is made of and 
recording what color they are. The quantitative researcher recognizes that it is only after 
obtaining a detailed and thorough account of the characteristics of the painting that the 
picture may be fully understood. 

A Review of Qualitative Methods 

Qualitative methods, in contrast to quantitative methods, investigate phenomena 
as they occur naturally, without researcher manipulation, dissection, or quantification. 

The information obtained, while less generalizable, provides a deeper understanding of 
the particular issue of interest and can identify issues sometimes hidden in structured 
quantitative research. Qualitative methods include interviews, focus groups, case studies, 
ethnographies, and historical analyses. Although quantitative methods are often useful to 
institutional researchers, sometimes the research question that needs to be answered 
requires a qualitative research method instead. Namely, qualitative methods are 
appropriate when information other than statistics and demographics is desired. A 
research problem involving non-returning students, for instance, could be studied 
quantitatively. The researcher could study the characteristics of students who left the 
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university versus those who continued their studies. Tests could be performed on the data 
to see if differences between the two groups were statistically significant. But if the 
researcher wanted to find out why the students decided to leave or remain at the 
university, a different type of investigation would be necessary. 

Researchers may opt to study historical texts and documents in their attempt to 
find out why students leave the university. This approach, which allows for virtually no 
interaction with individuals, relies on previously written materials to help explain why a 
phenomenon occurred. Historical research may uncover invaluable information - 
information that would not likely be attainable through other methods. A researcher, for 
instance, may locate multiple newspaper articles written about violent crimes in the area 
that were published in the same year that many female students left the university. 
Although it has the potential to place events in a meaningful context, historical research 
is rarely used by institutional researchers, perhaps because the issues they are interested 
in are happening concurrently with the study, or perhaps because of the heavy 
interpretive responsibility placed on the historical researcher. 

On the other hand, by conducting interviews, the researcher can ask the students 
why they left. These interviews would most likely be conducted via telephone since the 
students are no longer enrolled at the institution. Interviews allow researchers to obtain 
valuable and personalized information from a primary source. In addition, the interactive 
nature of the interview allows the researcher to ask probe questions to ensure that an 
adequate amount of information is secured and clarifying questions to assure that the data 
obtained is accurate. 

Focus groups are similar to interviews but allow researchers to obtain data from 
several individuals at once. Typically, focus groups consist of about eight to ten 
randomly chosen individuals who are asked to participate in the focus group because of 
their inclusion in a certain group. For instance, members for a focus group might be 
randomly drawn from all of the freshmen that left the university in a given year. 
Moderated and directed by a trained facilitator, focus groups are unique in that they allow 
a forum for open discussion about a specific topic. As a result, researchers may not only 
obtain responses to the questions they ask the participants, but may also accumulate 
knowledge based on the interactions among group members and the conversations and 
questions that ensue. 

Surveys, one of the hallmarks of institutional research, can be thought of as a brief 
interview with many people at once. Surveys can be used to gather both qualitative and 
quantitative data. The strength of survey research is its ability to obtain vast amounts of 
information in a relatively short period of time. The chief limitation of survey research 
may be its lack of depth. Whereas an interview can be thought of as the process of 
digging a very deep hole with a very small diameter, the survey can be likened to 
scratching the top of a very large surface. Although usually not as singularly focused as 
interview or group sessions, surveys have the potential to investigate phenomena in-depth 
through the use of open-ended questions. Currently, surveys are frequently employed 
when an institutional researcher aims to learn about the attitudes and opinions of student, 
employee, or alumni populations. 

Observation, in contrast, involves little or no interaction between the researcher 
and the participants. By allowing the researcher to observe and record events as they 
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naturally occur, without researcher manipulation or intervention, a genuine picture of 
reality can be obtained. As with all qualitative methods, however, researchers must be 
extremely open to what they find and must be vigilant in not allowing assumptions and 
biases to taint the data. 

Whereas the researcher engaging in direct observation maintains a certain 
distance from the individuals under investigation, ethnographers immerse themselves in 
cultures of interest. The ethnographer, then, observes and experiences events and 
phenomena from within the context in which they normally take place. This inside 
position allows for a unique understanding not permitted by other research methods. 
From our experience, ethnography appears to be rarely if ever used in institutional 
research although it may be an interesting avenue to pursue in future endeavors. 

Once the qualitative researcher has obtained a valid and reliable data set, an 
important next step is to make'sense of the findings. This stage is perhaps the most 
challenging one in qualitative endeavors. Not only is the researcher faced with an 
overwhelming amount of data, but also is responsible for drawing out meaningful and 
useful information. Unlike quantitative research methods that rely on the researcher’s (or 
their computer’s) statistical prowess to analyze numerical data sets, qualitative methods 
rely on the expertise and insight of the researcher to synthesize and understand textual 
data. The results of qualitative research, then, are primarily dependent on procedures 
internal to the researcher. 

Through content analysis, the researcher organizes the data into broader themes 
and issues. The qualitative researcher searches for structures and patterns and makes 
inferences based on these regularities. In qualitative research, the goal of analysis is to 
reorganize the data into categories that allow for comparisons within and between these 
categories and to develop possible theories. Computer software such as CETA, 
Ethnograph, HyperQual, and QualPro assist the researcher with qualitative data analysis, 
t)^ically employing a content analysis technique to code textual data. They are the 
qualitative counterparts to statistic^ programs such as SPSS and SAS and are often 
compatible with such programs. Qualitative data analysis software allows the researcher 
to conduct statistical analyses on non-numerical data. One of the advantages of 
qualitative data analysis software, like its quantitative counterpart, is the time it saves the 
researcher. Another strength is the program’s neutrality; a computer program is less likely 
to insert biases into the analyses than a human being who is personally analyzing the 
data. 

Regardless of the specific method used, the analytic process in qualitative 
research is contingent upon a capable researcher who has the ability to understand what 
individuals are trying to communicate. In this aspect, qualitative research can be 
compared to a painting hanging in an art museum that appears to consist of nothing more 
than thousands of independent dots of paint. Once the viewer steps back and looks again 
at the overall painting, however, a picture emerges. In this sense, it is only after one 
comes to understand the context of the bigger picture that the dots themselves are given 
meaning. The qualitative researcher loses the information associated with the number of 
dots, type of dots, and materials used in the painting, but gains an appreciation for larger 
patterns and holistic meaning. The qualitative researcher, then, must have the ability to 
look at an overwhelmingly large amount of seemingly separate data points and see the 
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picture they form. 



How are Research Methods Integrated? 

Mixed methodology research begins with the research issue or problem. From that 
point, a research design strategy is developed that includes at least two but sometimes 
many sets of data. Importantly, these data must be multidimensional in the manner of 
their collection, comprehensive to include possible divergent perspectives on the 
problem, and investigative to the point that the research may uncover "sleeping" or 
hidden facts or perceptions that impact the issue. The results of such research go beyond 
mere data tables or graphs. It is important that a report be generated and dialogue 
initiated that present the multiple findings and synthesize meaning to most fully represent 
issues that face institutions (Tashakkori and Teddlie, 1998). 

Using Multiple Methods and a Single Data Set 

There are three possible ways to apply multiple methods to a single data set. One 
is to use two or more different quantitative techniques to extract meaning from the data. 
Let’s look at an example where an institutional researcher is assisting a faculty member in 
analyzing their teaching methods in relationship to student performance. The data set in 
this example consists of an ability measure (e.g. SAT scores) and course grades in two 
different sections where the faculty member applied two different teaching methods. The 
data could be analyzed in two different quantitative ways. One could be to analyze the 
variance in grades within and between the two groups to determine if they significantly 
differed when controlling for the ability measure (ANOVA or ANCOVA). A second way 
could be to use a regression analysis to determine the linear relationship between ability 
measure and grade, and then compare this relationship between the two groups. 
Additional applications of quantitative methods exist for this data set, each yielding its 
own unique information. 

Using the same example, let’s explore the use of quantitative and quditative 
methods. At the end of the semester, the faculty member administered an assessment 
instrument that measured the students’ perceptions of the teaching methods through 
narrative responses to questions. A quantitative technique may be to categorize the 
students’ responses into predetermined categories, number them, and then compare 
differences between groups based on the two teaching methods (maybe using a T-test). A 
qualitative analysis may involve a content analysis of the narrative responses in order to 
explore emerging issues identified by students and to investigate whether patterns in 
those issues differed in nature between the two classes. 

Finally, a data set could be analyzed by multiple qualitative methods. Using the 
same data from the narrative responses above, a first analysis may be used to explore 
emerging issues as before. A second analysis may be designed utilizing information 
obtained in the first to further explore the teaching methods. For example, the identified 
emerging issues could be used to profile students within the two classes to determine 
whether these methods appeared to differently impact different types of students. 

Using Multiple Methods and Multiple Data Sets 

Mixed methodology research becomes even more potent, however, when more 
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than one data set is put to use. While only three possibilities exist when mixing 
quantitative and qualitative methods for any one data set, infinite possibilities exist when 
using these methods on more than one data set. While it is beyond the scope of this paper 
to detail the many types of mixed methodology research (see Tashakkori and Teddlie, 
1998), let’s look at the prior example of the faculty member teaching methods to show the 
increased explanatory potency of mixing methods with multiple data sets. 

One possible design for a mixed method approach to the research problem could 
consist of mixed quantitative and qualitative methods using the ability measures, course 
grades, and narrative responses. Initially, three quantitative analyses could be conducted 
to include statistical group comparisons in grades, in ability measures, and in categories 
of narrative responses. This could be followed by qualitative content analyses of narrative 
responses and identification of emerging issues. The difference between this multimethod 
approach and those described in the last section stems from the integration of findings 
from the two data sets. The results must be synthesized by the researcher before they are 
communicated to the community concerned with the issue. 

Researchers have noted the possibility of "paralysis by analysis" when multiple 
computations lead to a near stoppage of a project. This is obviously a caution, but using 
multiple data sets can help to ease this concern. Increasing available data rather than 
increasing analysis will help to add perspective to the project, diversify data sources, and 
allow for facts to be uncovered that may have been missed with fewer data sets. 

Following Through 

The critical and final phase of any research project is writing the report and 
communicating the results. This phase becomes even more important in mixed 
methodology research. The consuming task of this phase is integrating and synthesizing 
the results and meaning that are developed from multiple methods. Analyzing and writing 
the results given a singular data set is a rather straightforward task for an experienced 
researcher. Techniques such as analyses of variance, regressions, and content analyses 
have recognizable patterns in their narratives that can be replicated with different data; 
but when multiple methods are used with more than one data set, the task changes. Now, 
holistic meaning of the phenomena at hand becomes important. There are no pre- 
established routines for synthesis; researchers find themselves constructing the reality 
that is best portrayed by the results. 

Often the final step to single data set research is the production of the research 
report. The report can be sent out to interested parties who can read and understand the 
findings. In mixed methodology research, this technique must be modified. Since the aim 
of the report is to construct a reality based on the information, follow-up is necessary to 
facilitate the sharing of this reality by those concerned with the issue. Understanding their 
perspectives and concerns become relevant and the presentation and dialogue regarding 
the report become paramount. Mixed methodology research designed to lead to campus 
transformation should not end with the production of the report, it should continue with 
dialogue that leads to a shared understanding of the meaning surrounding the original 
research issue. 
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An Example of Successful Mixed Methodology Research 

The problem 

One example of the effective use of mixed methodology research was a recent 
study conducted at Shippensburg University. An external auditing office noticed an 
exception with the awarding of graduate assistantships with some inequity in tuition 
awards. The practice was to award a full tuition waiver to all students who had a graduate 
assistantship. The students would then work a certain number of hours for an hourly 
salary. Over time, the number of required hours began to vary and two types of 
assistantships, full-time and part-time, emerged. Both types, however, received a full 
tuition waiver. An inequity existed in that full-tuition waivers were given to students who 
may have only qualified for a half-time waiver. 

Design Strategy and Conducting the Research 

A multimethod design was constmcted that included quantitative data from the 
student record database, quantitative data from the human resources salary database, 
quantitative and qualitative data from a survey of graduate assistants and their 
supervisors, and qualitative data from interviews with campus personnel. Initially, the 
office of Institutional Research and Planning conducted several quantitative analyses on 
the program’s cost and the workload that the graduate assistants in the program 
contributed to the university. Guided by the goals of minimizing cost to the university, 
maximizing benefit to the student, and maximizing the students’ workload contribution to 
the university, linear programming was used to develop six possible scenarios. (Linear 
programming is a quantitative method with the objective of maximizing or minimizing 
some quantity given a number of boundaries or constraints.) The scenarios varied the 
number of graduate assistantships, the number of hours required by the assistantships, the 
amount of the tuition waiver, and the hourly salary. These data, however, were not 
sufficient to fully address the problem. 

A survey assessed student attitudes and opinions regarding the impact of the six 
scenarios. In addition, those departments that employed graduate assistants were 
surveyed regarding their preferences for the scenarios. The results from the surveys were 
analyzed using descriptive analytical techniques (quantitative) on the likert scale 
questions and using content analyses (qualitative) on the open-ended questions to isolate 
emergent issues. 

Synthesizing Data 

The final synthesis combined the results from these mixed methods into possible 
solution scenarios based on the utility of the quantitative models and the perceived 
preferences based on the attitudes and opinions of those involved. From the financial 
perspective, the quantitative data showed that one scenario was clearly superior in that it 
lowered costs to the university and maintained an adequate generation of workload by the 
graduate assistants. Student and employer opinion, however, showed this scenario to be 
rather unfavorable because graduate student enrollment might decline. Combining the 
results showed that a more moderate financial scenario was more highly favored that 
provided a lower cost than the present policy, maintained the necessary workload, and 
removed the inequity between full-time and part-time graduate assistants. 
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Presentation 

The report was discussed individually with executive managers with the 
explanation of the quantitative advantages of the scenarios and the perceived impact each 
would have based on the survey results. Specifically, the discussions focused on the 
optimal costs of the program to the institution, the contributions to workload that the 
graduate assistants provided, the financial support that the program provided to graduate 
students, perceptions of the graduate assistants’ supervisors, and the perceived popularity 
and attractiveness of the program to potential and continuing students. Further issues 
emerged from these informal interviews and the report was modified to include additional 
information. The final report was presented to a group of executive managers who 
engaged in dialogue on the issue. One of the recommended scenarios was selected and a 
new policy was written based on the research. Importantly, the new policy was 
financially viable, satisfied both students and employees, and appeared reasonable to 
management. 
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INTRODUCTION 

The current research interest in grading practices was triggered by a mounting 
concern over grade inflation in American educational system (Zangenehzadeh, 1988; 
Summerville et al., 1990; Franklin et al„ 1991; Agnew, 1993; Hensley, 1993; Farley, 
1995; Arenson, 1997; Yardley, 1997). A common understanding of the definition of 
grade inflation is that “students receive higher grades than their predecessors without 
a corresponding rise in achievement” (Yardley, 1997). 

This definition seems to have set the tone for most of the studies on grade 
inflation: first, many reseturchers went after the trend of grading patterns, trying to 
decide whether grades indeed increase over time; second, many researchers have 
focused their attention on the question of whether students actually learn more to 
deserve higher grades than their predecessors (Zangenehzadeh, 1988; Franklin et al., 
1991; Agnew, 1993; Hensley, 1993; Arenson, 1997; Scocca, 1998; Marklein, 1997a; 
Mullen, 1995). As a result, many have provided ample evidence to have successfully 
validated (e.g., Summerville et al., 1990; Farley, 1995) or dismissed (e.g., Adelman, 
1995; Olsen, 1997) the public suspicion of grade inflation. These research efforts 
have laid a solid foundation for further studies on this subject. 

However, a careful review of literature lead us to believe that there are at least two 
conceptual issues that have not been sufficiently addressed. First, the term grade 
inflation is problematic in a context that an objective standard is absent. Grades are 
measures of educational achievements, but they only make sense on a comparative 
basis. Comparisons can be made under unified or standardized conditions. The 
problem is, except for some nationally or internationally standardized tests (e.g., GRE 
and TOEFL) and various state-administered professional license examinations, 
classroom and non-classroom assessments are not standardized. Second, 
considerable amount of time and energy has been devoted to examine the correlation 
between student performance and their grades while they do not even participate in 
this measurement activity known as grading. In other words, since autonomy is a 
highly regarded value in higher education, grading will remain a faculty prerogative. 
The grading criteria and the factors affecting them would vary from campus to 
campus, from department to department, and even from classroom to classroom. 
Strictly speaking, what the grades tell us applies only to the students who are taught 
and tested exactly the same way. 

Therefore, this study was not designed to add another piece of testimony to the 
existing literature dismissing or validating the accusation of grade inflation. Nor do 
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the authors of this study have any intention to prove how well our students have done 
to deserve the higher grades, for the absence of absolute criteria will make this kind 
of arguments sound powerless. Instead, this study was designed to probe into the 
issue by asking what are the potential factors that would affect faculty grading 
practices. The purpose is to provide some necessary knowledge for public 
understanding and faculty awareness of the problem, and for policy intervention if 
this is ever deemed desirable. 

RESEARCH QUESTIONS 

In early 1998 in response to the request of a Board Committee of a large urban 
university system, the University Chancellor sent a memo to all the colleges asking 
for information regarding patterns of grading and grade distribution. Administrators 
at one of the University’ s colleges responded by conducting a series of formal and 
informal interviews with department chairs and faculty members and compiling 
grading data over the past ten years. Two conclusions were drawn from this 
preliminary probe: 1) grades have increased over the past ten years at the College, 
and 2) faculty grading practices, instead of student academic preparation or 
performance, are the source of the problem that needs to be addressed (Springer, 
1998). Consequently, some key factors were extracted from both grading data and 
interview results for an in-depth analysis. 

Since grading has always been considered to be a faculty prerogative (Kimmich, 
1998), it is natural to ask directly how instructors would evaluate students. Many 
faculty members interviewed indicated that they generally do not grade on a curve but 
rather mastery of the subject matter and performance of the students. “Experience 
over time determines faculty judgment of what constitutes mastery of subject matter 
and, consequently, the assignment of grades according to levels of perfoimance 
within college grading policies” (Mirrer, 1998). Therefore, it seems to be a 
reasonable assumption that faculty experience or seniority affects grade distribution. 
But we are not too sure about the direction of this hypothesis, since experience may 
help prevent grade inflation while the sense of security associated with tenure may 
also lead to ignoring college grading policies. 

There is another question as to whether the increased use of adjuncts may affect 
grading patterns (Mirrer, 1998). Specifically, there is a belief that adjuncts grade 
higher (Cheng, Hartman, Podell, & Zeldin, 1998). We would like, therefore, to 
examine the academic data as to whether there has been a difference between full- 
time and adjunct faculty in grading practices. 

For students, increases in grades may have to do with the pattern of course-taking 
(Kimmich, 1998). It has been suggested that students understand and are adept at 
“using the system” (Kimmich, 1998): grading patterns may be skewed when greater 
numbers of students opt for courses in which grades tend to be higher, or where the 
grading tends to be more subjective, such as those in the humanities, as opposed to 
courses in math and science, where the measures are more objective (Mirrer, 1998). 

In other words, grading patterns differ by discipline or department (Summerville et 
al., 1988; Cluskey et al., 1997). This is the third potential factor to be tested in the 
present study. 
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With grading data broken down by course levels (e.g., lower, upper, and graduate 
divisions), different grading patterns emerge. Therefore, the fourth factor we want to 
test is whether higher course levels associated with higher grades. 

There are many other hypotheses that are also worth formulating and testing. 
However, given the fact that most research projects on grade inflation are driven by 
the practical need of administrators to address concerns from their constituencies, this 
type of study is often ex post facto with data drawn from administrative databases. 
Oftentimes institutional researchers do not have the luxury of time and resources to 
conduct in-depth surveys. The present study was to demonstrate an institutional 
research effort that focuses on utilizing existing institutional data that can explain 
grading practices. Specifically, this article examines the following four research 
questions: 

(1) Do adjunct faculty award higher grades than full-time faculty? 

(2) Do junior faculty award higher grades than senior faculty? 

(3) Are grades generally higher in the humanities and social sciences than in 
science and technology disciplines? 

(4) Are grades generally higher in upper division courses? 

METHODS 
Data Sets 

Using one college as a case study, the empirical data were obtained from the 
campus-wide student information system. A working data set was constructed by 
extracting and combining data from different academic and administrative databases. 
The two main sources of data were the Course Masters File and the Course Card File. 

Designed as a preliminary study of the complex issue, the project was conducted 
as a cross-sectional study of various potentially important factors associated with 
grade distributions within the College. To validate the research results, this kind of 
“snapshot” approach to one semester’s data should be repeated for a number of times. 
The data analyzed in this study covered the fall semester of 1997. 

Recognizing the fact that students normally do not participate in grading decisions 
(with such exceptions as W’s, i.e., withdrawals), student identification and other 
characteristics are removed from the database. Meanwhile, the data file containing 
faculty characteristics such as their full-time/part-time status and ranking was merged 
with the main grade file. Using the summary function of database software (in our 
case, PARADOX), each grade is recorded as a separate “grading event” and the 
identical ones are summarized as “counts,” or the number of events. In the actual 
analyses, the variable “count” served as a “weight,” which is available in both SPSS 
and SAS. 

Analytic Procedures 

First, bivariate analytic procedures used in the study included Cross-tabulation and 
T-Test. Second, the techniques of “quasi-multivariate analysis” or elaboration (Chen, 
1998) were performed by applying statistical control where a relationship was 
suspected to be spurious in order to clarify the net effect of a potential causal 
influence. Finally, a multiple regression was conducted to verify the findings from 
the previous two steps. 
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Since we included all the grading events on the roll of the Fall semester of 1997, 
we did not need to make any statistical inference using these procedures. The 
inferential results would only make sense when the data were supposed to constitute a 
random sample. Yet in research practices, tests of significance are widely used to 
analyze nonrandom data, and some argue that significance at least points to the 
presence of a relatively considerable effect (Chen, 1998). The inferential results 
included in the following should only be interpreted in such a manner (i.e., for a 
h 5 ^othetical random sample of a larger population). 

RESULTS 

Collegewide Grade Distribution 

Altogether, there were 31,916 grades/grading events recorded for the Fall of 1997 
at the College. Table 1 breaks these grading events into three distinctive groups: (1) 
regular grades ranging from A to F, grouped into high, medium, and low/failing 
grades; (2) the grades of official and unofficial withdrawals and “incomplete;” and (3) 
Non-judgmental grades. Overall, excluding non-judgmental grades, close to 50 
percent of the grades awarded in Fall 1997 were on the higher end of the grading 
spectrum (B and up), nearly one-quarter of the grades, medium (C to B-), and 8 
percent, low/failing (D and F). 

Bivariate Analysis 

Full-Time vs. Part-Time (Adiunct) Faculty. A total of 594 faculty members were 
involved in grading and included in the study. Of the 594 faculty members, 218 
(36.7%) were full-timers, and 376 (63.3%) were adjuncts (part-timers). Full-time 
faculty were responsible for 15,440 grades/grading events, which account for 46.8% 
of the total. Adjunct faculty were responsible for 17,544, or 53.2% of the total 
grades/grading events. 

Table 2 clearly indicates that, measured by mean quality points per credit, adjunct 
faculty gave average grades 0.107 point higher than full-time faculty. Table 3 shows 
that adjunct faculty gave more high grades than full-time faculty (52.1% vs. 46.7%), 
and they gave fewer low grades than full-time faculty (7.4% vs. 8.6%). Row 
percentages are used in Table 3 to facilitate such comparisons. The results indicate 
that adjunct faculty give higher grades than full-time faculty. 

It is noticeable that while students withdrew officially from full-time faculty’s 
classes at a higher rate than that from adjuncts’ (10.3% vs. 8.1% of W’s), a higher 
proportion of students received a grade of unofficial withdrawal (WU) from adjuncts. 
In addition, full-time faculty seemed to be more willing to give an incomplete grade 
(5.4%) than adjuncts (3.8%). 

Faculty Rank/Seniority . Of the 218 full-time faculty members, 69 were full 
professors, 71 associate professors, 67 assistant professors, and 1 1 under other titles 
such as lecturers. Senior faculty (full and associate professors) were responsible for 
9,1 16 grading events, which account for 60.2% of the all the grades given by full- 
time faculty. Junior faculty (assistant professors and faulty with other titles) were 
responsible for 6,016, or 39.8%, of the subtotal of grades/grading events. 

Table 2 shows that, measured by mean quality points per credit, there was no 
significant difference between junior and senior faculty in assigning grades. In Table 
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3, a chi-square test confirmed the fact that no significant difference existing between 
junior and senior faculty in grading practices. 

Table 1. Grade Distribution and Grouping 



Quality Points 



Grades and 
Grouping 




per Credit 


Frequenc 

y 


Percent 


Regular Grades 


High 


A 


4.0 


5,115 


16.50% 




A- 


3.7 


2,916 


9.40% 




B+ 


3.3 


3,362 


10.80% 




B 

Subtotal 


3.0 


3,970 

15,363 


12.80% 

49.60% 


Mediu 


B- 


2.7 


2,267 


7.30% 


m 




C+ 


2.3 


1,992 


6.40% 




C 

Subtotal 


2.0 


2,918 

7,177 


9.40% 

23.20% 


Low 


D 


1.0 


1,602 


5.20% 




F 

Subtotal 


0.0 


878 

2,480 


2.80% 

8.00% 


Grades in 
Question 




W - Withdrawal 


N/A 


2,827 


9.10% 




WU - Unofficial 
Withdrawal 


0.0 


1,743 


5.60% 




I - Incomplete 
Subtotal 


N/A 


1,406 

5,976 


4.50% 

19.30% 


Non-Judgemental Grades* 


N/A 


920 


N/A 



Total 31,916 



* Including grades assigned to auditor, administrative 
withdrawal, etc. 



Table 2. T-Test of Numbered Grades of "A" to "F‘ 



Variable 


Count 


Mean 


Standard 

Deviation 


Mean 

Difference 


F* 


Part-Time 


13,434 


2.943 


0.979 






Full-Time 


1 1 ,586 


2.836 


1.018 


0.107 


34.478** 


Senior 


6,882 


2.836 


1.005 






Junior 


4,704 


2.838 


1.037 


-0.002 


0.537 


H&SS 


15,016 


2.930 


0.976 






S&T 


9,102 


2.817 


1.055 


0.113 


105.727** 



* Levene’s Test for Equality of Variance. ** p=.000. 



Table 3. Chi-Square Tests of Grading Groups 









Grading Group 












High 


Medium 


Low 


W 


WU 


1 


Total 


Chi-Sq* 


Part- 

Time 


52.1% 


22.5% 


7.4% 


8.1% 


6.0% 


3.8% 


16,366 




Full- 

Time 


46.7% 


23.8% 


8.6% 


10.3 

% 


5.2% 


5.4% 


14,630 


159.29* 

* 


Senior 


45.8% 


24.3% 


8.3% 


10.7 

% 


5.3% 


5.5% 


8,773 




Junior 


48.1% 


23.1% 


9.1% 


9.6% 


4.9% 


5.2% 


5,857 


15.23 


H&SS 


51 .0% 


23.0% 


7.4% 


7.4% 


6.0% 


5.2% 


18,462 




S&T 


46.8% 


23.3% 


9.6% 


12.2 

% 


5.1% 


3.1% 


1 1 ,426 


328.308 

** 


Lower 


46.3% 


24.5% 


8.9% 


10.0 

% 


6.3% 


4.0% 


25,934 




Upper 

* 


62.9% 


18.7% 


4.0% 


5.2% 


2.6% 


6.6% 


4,248 


592.083 

** 



* p<.01. 

DF=5. 






Disciplinary Difference . Academic disciplines or departments at the College are 
organized in two broad divisions: the Division of Humanities and Social Sciences 
(H&SS) and the Division of Science and Technology (S&T). In Fall 1997, 19,069 
grading events took place in the Division of H&SS and 1 1,649 in S&T. The T-Test 
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in Table 2 points to the fact that, measured by mean quality points per credit, student 
average grades were 0. 1 1 3 point higher from the courses in the Division of H&SS 
than those in S&T. Table 3 shows that the H&SS Division was responsible for 51.0% 
of the high grades awarded, whereas S&T, 46.8%. On the other hand, H&SS’s low 
grades accounted for 7.4% of the total, while S&T’s accounted for 9.6%. The results 
show that grades are higher in the humanities and social sciences than in science and 
technology disciplines. 

It is interesting that, while the faculty in the H&SS Division gave more unofficial 
withdrawals (WU’s) and incomplete grades (I’s) than S&T faculty did (6.0% vs.5.1% 
and 5.2% vs. 3.1%, respectively), the latter received far more W’s from the students 
(7.4% vs. 12.2%). It seems that, though S&T faculty is less likely to “inflate” grades, 
it might be of greater concern in terms of a need for pedagogical improvements to 
help students overcome the difficulties. 

Course Levels. Given the fact that the College’s academic offerings range from 
associate degree programs all the way to the Masters, the frequencies of 
grades/grading events by course level are pyramidal: the higher the course level, the 
fewer the students/grades. Table 4 displays an unambiguous pattern: the higher the 
course level, the higher the average grades. 

With undergraduate courses selected, a chi-square test was performed and the 
result (see Table 3) confirms a significant grading difference between lower level 
courses (100- and 200-levels) and upper level courses (300- and 400-levels; 500-level 
courses are excluded for a more rigorous test). Upper level instructors gave out 
62.9% high grades, as opposed to the lower level, 46.3%. Meanwhile, upper level 
instructors gave less than one-half of low grades as compared with lower level 
instructors (4.0% vs. 8.9%). What is especially intriguing is that while upper level 
instructors seemed to be more prepared to award incomplete grades (6.6% vs. 4.0%), 
they assigned or received by far the fewer WU’s and W’s (5.2% vs. 10.0% and 2.6% 
vs. 6.3%, respectively). This suggests an important difference between incompletes 
and withdrawals. 

Table 4. Mean and Standard Deviation of Numbered Grades of "A" to "F" 



Course 

Level 


Count 


Mean 


Standard 

Deviation 


1 00-Level 


3,652 


2.736 


1.076 


200-Level 


7,025 


3.006 


0.879 


300-Level 


2,658 


3.125 


0.856 


400-Level 


979 


3.188 


0.798 


500-Level* 


125 


3.689 


0.498 


600-Level 


480 


3.469 


0.584 


700-Level 


84 


3.607 


0.560 


800-Level 


17 


3.706 


0.588 


Total 


25,020 


2.894 


0.999 



* Including independent study, internship, and special topics. 



Elaboration 

Table 3 suggests that full-time and part-time faculty graded differently while there 
was no significant difference between senior and junior faculty. These two findings 
were further tested under more controlled conditions to make sure that the differences 
found are not spurious. The logic is that if the said differences disappear or weaken 
after controlling for the other variables, then the differences may be to some degree 
spurious. If the differences stay unchanged after controlling for the other variables, 
then they are probably true or nonspurious (Chen, 1998). 

For the categorical data presented in Table 3, statistical control was carried out via 
a partial- or sub-table approach. Tables 5 and 6 present the results of the elaboration. 
A consistent pattern of full-/part-timer difference in grading practices controlling for 
disciplinary difference and course levels suggests that the results of the bivariate 
analysis presented earlier are probably true (i.e., nonspurious). However, the 
conclusion regarding the difference between senior and junior faculty in grading 
practice can be partly attributed to the disciplinary difference because the finding is 
reversed for grades awarded in the H&SS lower level and S&T upper level courses. 
That is, senior faculty teaching lower level H&SS courses tended to award slightly 
more higher grades than their junior counterparts, while junior faculty teaching S&T 
upper level courses tended to do the same than their senior counterparts. 

Multivariate Analysis 

A regression procedure was conducted to provide a comprehensive understanding 
through multivariate analysis (Table 7). The results reconfirmed the influences of 
course level and disciplinary differences on faculty grading, and course level had a 
greater impact than disciplinary difference. Adjunct faculty graded higher on average 
than full-time faculty, whereas junior faculty do not seem to have graded higher than 
their peers in senior ranks. 



Table 5. Grade by Faculty Full-/Part-Time Status Controlling for Course Level 
and Discipline 



Course 

Level 


Discipline 


Grade 


Faculty Status 
Part Time Full-Time 


Chi-Square 


Lower 


S&T 


High 


60% 


51% 








Medium 


30% 


32% 








Low 


10% 


17% 


82.967* 




H&SS 


High 


62% 


55% 








Medium 


28% 


34% 








Low 


10% 


11% 


59.220* 


Upper 


S&T 


High 


77% 


66% 








Medium 


19% 


25% 








Low 


4% 


8% 


17.306* 




H&SS 


High 


84% 


70% 








Medium 


15% 


26% 








Low 


1% 


5% 


61.388* 



*p<.01. 
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Table 6. Grade by Faculty Junior/Senior Status Controlling for Course Level 
and Discipline 



Course 

Level 


Discipline 


Grade 


Faculty Status 
Junior Senior 


Chi- 

Square 


Lower 


S&T 


High 


53% 


50% 








Medium 


31% 


33% 








Low 


16% 


17% 


2.221 




H&SS 


High 


53% 


57% 








Medium 


34% 


34% 








Low 


13% 


10% 


21.850* 


Upper 


S&T 


High 


75% 


62% 








Medium 


15% 


30% 








Low 


9% 


8% 


26.841* 




H&SS 


High 


70% 


69% 








Medium 


24% 


27% 








Low 


6% 


4% 


2.503 


P<.01. 



Table 7. Multiple Regression Results on Numbered Grades of "A" to "F" 
Beta t Coding Scheme 


Course 


0.173 


26.527 


1=1 00-level, 4=400-level, 


Level 




* 


5=graduate 


Adjunct 


0.084 


1 1 .246 
*■ 


1=adjunct; 0=not adjunct 


Division 


0.038 


5.938* 


1=H&SS; 0=S&T 


Junior 

Ff 


0.014 

0.034 


-1 .928 


1=junior faculty; 0=not junior 



* p<.05. 
DISCUSSION 



College administrators often find themselves caught in a dilemma when their 
college is being accused of grade inflation, especially when “hard” data over time 
seem to support the accusation. On the one hand, since grading is always a faculty 
prerogative, the administration is supposed not just to refi-ain from interfering faculty 
grading practice but to defend this basic academic freedom. On the other hand, 
institutions, especially the public ones, are increasingly held accountable for their 
performance and outcomes, and nothing serves as a more negative indication of a 
college’s lack of academic standards than grade inflation. Therefore, to college 
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administrators, this is not a matter of whether to intervene with faculty grading 
practices or not; it’s a matter of how. 

Past research has shown that to simply compile data or to go after the trend of 
change in grading patterns over time, as most researchers have done so far, does not 
help solve the problem at all. It is our belief that the judgment of whether there is 
grade inflation is more of a normative or political issue rather than an academic or 
scientific one. In other words, it is the lack of unified or standardized criteria in 
classroom grading that makes it impossible to speak about grade inflation in any 
absolute terms. Therefore, in the last analysis, to understand the potential factors 
contributing to the variation in grade distribution becomes a prerequisite for any 
effective policy intervention, currently represented by a desire to keep grades in 
check or to achieve grade deflation (Agnew, 1993). 

The findings of this article suggest that greater attention should be paid to upper 
level courses, courses offered in the humanities and the social sciences, and part-time 
faculty grading practices. Faculty rank is generally not a concern, though senior 
faculty teaching lower level courses in H&SS and junior faculty teaching upper level 
S&T courses tended to grade higher. This is the approach that identifies possible 
problem areas without confirming or dismissing the accusation. The results provide 
administrators with very specific and in-depth knowledge about faculty grading 
practices at the college. A study of this nature is to guide college administrators in 
making policies that target specific areas of problem without having to come up with 
any sweeping changes that may hurt the innocent. 
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DESIGNING GRADUATE ADMISSION STUDIES TO INFLUENCE 
CAMPUS TRANSFORMATION 



Anne Marie Delaney 
Director of Institutional Research 
Babson College 

Introduction 

Purpose . This paper presents the rationale, methodology and selected results from 
a study which examined the enrollment decision processes of students accepted to a 
newly redesigned, two-year Master of Business Administration (MBA) Program. The 
immediate purpose of this study was to acquire an in-depth understanding of the 
motivation and enrollment decision processes of accepted students; to obtain information 
about the School’s image among these students; and to identify the School’s competitive 
position in the marketplace. The ultimate purpose was to use this information to design 
program modifications and craft recruitment strategies that would successfully attract the 
optimum number of high quality students. 



Program Description . This study was sponsored by and conducted for the F.W. Olin 
Graduate School of Business at Babson College. The Olin Graduate School is committed 
to being an internationally recognized leader in graduate management education. To 
achieve this goal, the school recently created and implemented an innovative curriculum 
designed to prepare graduates for entrepreneurial leadership in a changing environment. 
Reflecting this theme, the curriculum focuses on the following learning outcome areas: 
team-based learning; pragmatic thinking; a cross-functional perspective; a business-wide 
perspective; persuasive communication; and a global opportunity orientation. Field- 
based learning is a vital and defining characteristic of this curriculum. 

Review of the Literature . Three streams of literature provide a theoretical background 
and research base for the present study: first, the extensive body of research examining 
undergraduate students’ college choice process; second, research on graduate students’ 
enrollment decision process; and third, specific studies focused on the enrollment decision 
process of graduate business students. 

Undergraduate College Choice Studies. At the undergraduate level. Chapman 
(1981) identified structural and attribute variables, such as demographic 
characteristics, college characteristics and financial aid needed as predictors of 
college choice. Hossler and Gallagher (1987) proposed a three stage model of college 
choice: a predisposition stage, a search phase; and a final selection stage. In terms of 
specific variables, research from the 1960’s to the present has consistently established 
the effect of family income on students’ college choice (Baird, 1967; Flint, 1992); 

Hearn, 1984, 1988; Zemsky & Oedel, 1983). Severed undergraduate level studies have 
also discovered strong relationships between students’ ability and the college choice 
process. These studies consistently revealed that high ability students attribute 
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primary importance to academic factors (Fink, 1997; Galotti & Mark, 1994; Wanat & 
Bowles, 1992). 

Undergraduate college choice studies have also established the relationship 
between students’ perception of institutional characteristics and their college choice. 
Examples include the quality of staff/faculty, types of degree programs, faculty 
student interaction, and financial assistance (Coccari & Javalgi, 1995); good academic 
reputation, good job placement, and well managed facilities (Comm & LaBay, 1996); 
and career preparation, distance from home, the quality of the school’s research 
program, and library resources (Martin’s, 1996). Also on the perceptual level, Terkla 
and Pagano (1993) demonstrated the relevance of the institution’s image to students’ 
enrollment decision. 



Graduate School Admission Studies . Kallio (1995) reports that graduate student 
decisions are affected by some of the same factors influencing undergraduate students: the 
academic reputation of the institution; program quality and size; price; financial aid; 
geographic location; contact with faculty; and individual student characteristics, such as 
academic ability and achievement. Graduate students appear to differ from undergraduates 
in terms of a greater influence of spouse, family and work considerations. Olson and King 
(1985) offer a preliminary model of graduate students’ college choice that specifies three 
major factors influencing initial consideration of a graduate school - geographic location, 
personal contact with the faculty; and the reputation of the department, and three additional 
factors as determinants of the final decision: a positive interaction with university 
personnel, personal reasons, and previous undergraduate attendance. In an earlier study of 
over 1,000 new graduate students, Malaney (1987) found that students pursued graduate 
education more frequently to fulfill their desire to leam more and to achieve personal 
satisfaction rather than to accomplish job related goals. 

Business School Admission Studies . Research on the choice process of business and 
management students is most pertinent to the present study. Stolzenberg and Giarrusso 
(1987) reported that career entry or mobility was the most frequently reported reason for 
pursuing an MBA. However, respondents at schools with highly competitive admission 
processes were more likely than others to emphasize the development of management skills 
and business knowledge as their primary reason for pursuing the MBA degree. McClain, 
Vance and Wood (1984) found that investment of the school’s resources, in terms of time 
and financial aid resources, had the most significant positive effect on students’ final choice 
of a graduate business school. Webb’s (1992) research, with approximately 1500 graduate 
business students, identified academic reputation and accreditation as the two most 
important fixed college characteristics; the availability of evening classes and the quality of 
the program as the two most important program characteristics; and the potential 
marketability of the degree as the only marketing factor perceived to influence business 
students' college choice decisions. 
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Methodology 



Data Collection . Data for this study were collected by means of a mailed survey 
designed by the author in collaboration with the Graduate School Dean and Director of 
Graduate Admissions. The survey was administered to 334 students accepted for the fall 
1997 two-year MBA Program. A response rate of 71 percent was achieved, 94 percent 
for enrolling students and 55 percent for non-enrolling students. 

Description of the Sample The respondent group included 238 accepted students; 55 
percent enrolled and 45 percent did not enroll at our school. In terms of demographic 
characteristics, 67 percent, were male and 63 percent were single. The mean age was 28. 
Close to 40 percent reported annual incomes between $20,000 and $40,000 and 30 
percent reported incomes ranging from $40,000 to $60,000. The majority of these 
accepted students had some work experience prior to applying for an MBA program. 
These students are academically superior; 70 percent earned scores of 600 or higher on 
the Graduate Management Admission Test (GMAT). 

Results 

Reasons for Pursuing an MBA. As shown in Figure 1, results from this research 
identify three primary reasons motivating students’ decisions to pursue an MBA degree. 
Some75 percent identify career advancement as their most or second most important 
reason for pursing an MBA degree. Another 66 percent report personal fulfillment as 
their most or second most important reason, and 50 percent of the accepted student 
population cite career change as their primary or secondary reason for pursuing an MBA. 

Results presented in Figure 1 bear implications both for recruitment strategies and 
curriculum development. Since these data reflect students’ primary motivation for 
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pursuing an MBA, further research might investigate what knowledge and skills are 
required for students to achieve career advancement; in what ways students are seeking 
personal fulfillment through the MBA Program and what are the typical career changes 
prospective students are seeking to achieve. Information on these issues would be 
extremely useful in guiding program revisions to support students’ goals. 

Importance of Graduate School Characteristics . Since the characteristics of graduate 
schools also influence students’ choice, survey respondents were asked to rate the 
importance of 17 graduate school characteristics in relation to their choice. Results 
revealed that the institution’s academic reputation, the quality of teaching and the value 
of the degree from the school hold a very high level of importance in students’ choice of 
a graduate school; 92, 87, and 87 percent respectively rated these chciracteristics as ' very 
important Next in order, between 50 and 70 percent rated the following five factors as ' 
very important quality of enrolled students, internships opportunities, program structure 
and requirements, the faculty’ s reputation and diversity of course offerings. It is 
interesting to note that the factors which hold the highest level of importance relate to 
essential aspects of the graduate degree program. In contrast, issues of much less 
importance to students include on-campus housing, employer tuition reimbursement, job 
availability and educational plans of spouses; fewer than 10 percent rated these factors as 
' very important ' in their choice of a graduate school. 

Chi square analyses identified statistically significant differences between enrolling 
and non-enrolling students on three of the 17 characteristics: size of the school, program 
structure and requirements, and opportunities for friendship. Some 18 percent of the 
enrolling students, compared with 8 percent of the non-enrolling students, rated size of 
the school as ' very important ' to their choice (X^= 6.20, p < .05). Also, 68 percent of 
the enrolling students, compared with 47 percent of the non-enrolling students, rated 
program structure and requirements as ’ very important ' to their choice (X^= 13.02, 
p < .001). Finally, opportunities for friendship appeared to be more important for 
enrolling students; 42 percent of the enrolling students, compzired with only 32 percent of 
the non-enrolling students, rated this aspect as ' very important ' to their graduate school 
choice (X^= 5.98, p < .05). These results suggest that enrolling students preferred the 
smaller size of our school to the larger competitor institutions. They also valued the 
innovative program and the collegial environment as conducive to forming friendships. 

Top Competitor Graduate Schools . In addition to understanding the factors that 
influence students' choice of a graduate school, it is also important to ascertain the 
graduate schools with which one is competing for the largest number of accepted 
students. Therefore, accepted students were asked to identify the top five schools to 
which they applied; where they were accepted; and where they were offered financial aid. 
The administrative report presented two sets of competitors. The first, top application 
competitor set specified the ten graduate schools with which our school shared the largest 
number of applications. The second, top alternative choice competitor set identified the 
graduate schools most frequently chosen by the non-enrolling students and the schools 
reported by the enrolling students as their top alternative choice to our school. 
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Further analyses included rankings of these top competitors by number of 
applications or choices among all accepted students as well as among enrolling and non- 
enrolling students. The top alternative choice schools were also classified in two 
subgroups: those with which we yielded more than 50 percent of accepted students and 
those with which we yielded less than or equal to 50 percent of accepted students. 
Subsequent analyses were conducted to determine how our school differed from these 
sets of competitor institutions in terms of accepted students’ evaluation of the admission 
process and their ratings of various enrollment decision factors. 

Evaluation of the Admission Process. Survey respondents were asked to evaluate 
several sources of information and experiences encountered during the admission process 
both at our school and at alternative choice graduate schools. Aspects of the admission 
process specified include publications, organized admission activities, and contact with 
significant individuals, such as students and faculty. Table 1 identifies the admission 
experiences with significantly different ratings for our school compared with alternative 
choice graduate schools. These are the second choice schools for our enrolling students 
and the schools at which non-enrolling students matriculated. 



Table 1. Signiflcant Differences Between Accepted Students’ Ratings of Admission 
Experiences at Our Graduate School and Alternative Choice Graduate Schools 
Mean Ratings 



Admission Experience 


F.W. Olin 


Alt. Choice 


Mean Diff. 


t Ratio 


Promptness of Reply to Requests 


3.46 


3.11 


0.35 




Individual Attention 


3.51 


3.19 


0.32 


3 33*** 


Telephone Contact with Admissions 


3.42 


3.14 


0.28 


1 . 16 ** 


Correspondence with Admissions 


3.51 


3.23 


0.28 


3.07** 


Visit to Campus 


3.55 


3.34 


0.21 


2.00* 


Graduate School Guide Ratings 


3.35 


3.52 


-0.17 


-2.29* 


Financial Aid Communication 


2.81 


3.18 


-0.37 


-2.29* 



* p < .05 ** p < .01 *** p < .001 



As illustrated in Table 1, the mean ratings are higher for our graduate school on five of 
the seven admission experiences. These superior ratings reflect a high level of satisfaction 
with promptness of reply to requests, individual attention, correspondence and telephone 
contact with the admission office, and the visit to campus. In contrast, the two admission 
experiences with superior ratings for the alternative choice schools are graduate school 
guide ratings and financial aid communication. 
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Influence of Enrollment Decision Factors . Survey respondents were asked to rate 
our school and their alternative choice school on several enrollment decision factors that 
students typically consider in their choice of a graduate school. These factors included 
academic issues, such as academic reputation and access to faculty; financial factors 
concerning tuition and expected cost after financial aid; social factors relating to social 
activities, and the quality and diversity of the student body; and personal considerations 
regarding distance from home and spouse’s preference. Results from t test analyses 
identified statistically significant differences between accepted students’ ratings of 
enrollment decision factors for our school and alternative choice graduate schools on 13 
of 27 factors. Seven of these differences involve superior mean ratings for our school. 

Table 2. Significant Differences between Accepted Students’ Ratings of Enrollment 
Decision Factors for Our School and Alternative Choice Schools 



Mean Rating 



Enrollment Decision Factor 


F.W.Olin 
Grad. School 


Alternative 

Choice 


Mean 

Difference 


t Ratio 


International Study Programs 


3.45 


2.64 


.81 


7 20*** 


Field-Based Programs 


3.45 


2.74 


.71 


6.69*** 


Program Structure/Requirements 


3.37 


2.91 


.46 


5.06*** 


Team-Based Learning 


3.44 


3.04 


.40 


4.58*** 


Access to Faculty 


3.40 


3.06 


.34 


3 "73*** 


Class Size 


3.11 


2.83 


.28 


3 3q*** 


Specific Academic Programs 


3.37 


3.12 


.25 


2.64** 



*p<.05 **p<.01 ***p<.001 



These data reflect a very high regard for our graduate program. All of the statistically 
significant, superior mean ratings refer to characteristics of the program including 
international study programs, field-based programs and the program structure and 
requirements. In contrast, all of the significantly higher mean ratings for the alternative 
choice graduate schools involved non-academic, programmatic issues. 

Enrollment Decision Scales . Table 3 presents the names, components and reliability 
coefficients for four enrollment decision scales. A student’s score on these scales is based 
on his or her mean response on the component items. As reflected in the reliability 
coefficients, the internal consistency for three of the scales are moderately strong. 
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Table 3. Description of the Enrollment Decision Scales 



Program Characteristics (r = .76)* 

Program Structure/ Requirements 
Area of Specialization 
Field-Based Programs 
Specific Academic Programs 



Academic Quality (r = .78) 
Academic Reputation 
Graduate School Faculty 
Graduate School Ranking 
Teaching Reputation 



Career Network (r = .77) 



Diversity/International Programs (r = .62) 
Diversity of the Student Body 
International Study Programs 



Alumni Network 
Post-Graduate Employment 
Quality of the Student Body 



* r = Chronbach alpha reliability coefficient 



Predicting Students’ Choice for the F.W. Olin Graduate School 



Discriminant analysis was employed to determine which combination of variables 
would predict accepted students ’enrollment decision. Results from bivariate analyses were 
used to select potential predictors for the discriminant analysis. Variables examined 
included demographic characteristics; educational background; financial aid status; 
students' ratings on the importance of various graduate school characteristics; and their 
ratings of our school on admission experiences and on several enrollment decision factors. 

Table 4 displays the discriminant function coefficients, or relative weights, for 
variables found to be significant predictors of students' enrollment decision. As shown, 
the strongest predictor is students' rating on program characteristics followed by student 
diversity and international programs, career network, graduate school guide ratings and 
academic quality. Students were significantly more likely to enroll at our school to the 
extent that they rated our school positively on these factors. The discriminant function 
including these five variables accurately predicted the enrollment decision of 77 percent 
of the respondents. The canonical correlation of .47 indicates that this function explains 
22 percent of the variance in accepted students' decision whether or not to enroll at out 
graduate school. 



Table 4. Discriminant Analysis Results: Predicting Students’ Enrollment Decision 



Predictors 


Standardized Discriminant 
Function Coefficients 


% Correctly 
Classified 


Program Characteristics 


.51 


77% 


Student Diversity & International Program 


.38 




Career Network 


.28 




Graduate School Guide Ratings 


.21 




Academic Quality of the Program 


.13 




Canonical Correlation 


.47 





=48.09; df=5;p<.001 



Discussion 

Dissemination and Utilization of the Research Findings 

Dissemination of the results from this research included a preliminary 
presentation of the findings to the Graduate School Dean and the Graduate Advisory 
Board followed by a complete report to the President, the President’s Cabinet, the 
Graduate School Dean, and the Graduate Admission Director. In addition, an executive 
summary was distributed to members of the Graduate Advisory Board which consists of 
24 high-level executives selected for their exceptional stature in the business community 
and their commitment to the College. Following the distribution of the reports, the author 
made presentations and engaged in discussions withthe Graduate School Dean, the 
Graduate Admission Director and staff, the Director of Career Services, and members of 
the Graduate Advisory Board. 

Impact of the Study 

Communication from the Graduate School Dean and the Director of Graduate 
Admissions verified that the results of the study were used not only to validate and 
enhance the recognized strengths of the Graduate School, but also to implement 
recommended changes for program improvement. As a result of the study, the Graduate 
School continued to make a strong personal investment in the admission program 
emphasizing individual attention, visits to campus, and the involvement of the Dean at 
information sessions. Recmitment strategies also continued to highlight the strengths of 
the program with its unique interdisciplinary, team-based and field-based learning 
experiences and opportunities for participation in a dynamic international program. 
Further, as a result of the study, the Graduate School developed strategies to improve 
financial aid communication and the MBA Admission Forum; to increase guidebook 
ratings; and to intensify the focus on career services. Finally, according to the Dean, the 
study strengthened the administration’s position relative to the design of career paths 
within the academic program. 
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Relationship to Previous Research 



Reasons for Pursing Graduate Study . Findings from this study confirm 
previous research results regarding the reasons business students pursue graduate 
education, but they differ somewhat from conclusions about graduate students in general. 
The vast majority of respondents in this study, 75 percent, reported career advancement 
as their most or second most important reason for pursuing an MBA degree. Another 66 
percent identified personal fulfillment and 50 percent cited career change as their primary 
or secondary reason for pursuing an MBA degree. Business students’ focus on careers in 
consistent with the research of Stolzenberg and Giarrusso (1987) who reported that career 
entry or mobility was the most frequently reported reason for pursuing an MBA. In 
contrast, Malaney (1987), who studied a group of new graduate students, found the desire 
to learn more as the most frequently reported reason for pursing graduate study, followed 
by the desire to achieve personal satisfaction. 

Reasons for Institutional Choice . Data from this study revealed that the most 
important factors in students’ choice of a graduate school were academic reputation, 
quality of teaching and value of the degree from this school. Similarly, Webb’s (1992) 
research with approximately 1500 graduate business students identified academic 
reputation as one of two most important fixed college characteristics and the potential 
marketability of the degree as the only marketing factor perceived to influence business 
students’ college choice decisions. 

Evaluation of the Admission Experience . Respondents’ evaluation of their 
admission experience reflected a very high regard for the personal and professional 
services offered by the graduate admission team. Their ratings and comments 
emphasized the importance of individual attention, promptness of reply to requests, 
communication with the admissions staff, as well as visit to campus and contact with 
enrolled students. In an earlier study, McClain, Vance and Wood (1984) also found that 
investment of the school’s resources, in terms of time as well as financial aid resources 
had the most significant positive effect on students’ final choice of a graduate business 
school. 



Enrollment Decision Factors . Clearly, the strongest determinant of our 
students’ decision to enroll was their positive evaluation of the program. Similarly, 

Kallio (1985), Olson and King (1985), and Webb (1992) also found that program quality or 
reputation of the department was a significant influence on students’ decision to choose a 
particular institution. Finally, the discriminant analysis model produced from this 
research highlights several variables found to be significant predictors of students’ 
enrollment decision in previous research. Examples include students’ evaluation of the 
program; perceived opportunities from a career network; and the value of the school’s 
degreel. 
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Conclusion . This paper illustrates how a graduate admission research study was used 
to evaluate the impact of an innovative graduate degree program and the effectiveness of 
admission strategies in recruiting students to this program. Results from this study were 
used to inform the development of strategic, policy relevant recommendations designed 
to increase the recruitment of high quality students; to enhance the effectiveness and 
relevance of the graduate degree program; and to further strengthen the School’s national 
and international reputation. 
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INTRODUCTION 

As part of a national grant awarded to the University that will document the effects of the 
baccalaureate research experience, researchers in the Office of Institutional Research & 
Planning were asked to coordinate an alumni survey. The primary purpose of the survey 
was to document alumni attitudes about the academic, personal, and career skills gained 
from the undergraduate experience. Specifically, we sought to obtain a matched sample 
of alumni, half of whom were involved in one or more research experiences as an 
undergraduate, and the other half who were not. To provide a more accurate comparison 
of research verses non-research students, we wanted to match the two groups on 
graduation year, undergraduate major, and academic performance, based on class rank 
within their major and GPA. 

HOW TO ACHIEVE THE RESPONDENT SAMPLE 

Once we received Human Subjects approval for distribution of the survey, we 
discussed how to select the sample. Typically, name and addresses of alumni are 
provided by the Office of Alumni Relations. Because this survey project required a 
matched sample, substantial thought was invested in how our staff could best obtain the 
sample. After looking at various options, we decided that the easiest, most efficient 
method was to manually select from the data file names for the matched group instead of 
trying to write a program in Natural, SPSS, or SAS. Access was chosen for the database 
management tool because of its ability to easily manipulate tables, create reports, produce 
high quality mailing labels, track the return of surveys, and interact directly with other 
programs, such as Excel. 

Table 1 lists the steps and fields used in Access to achieve the respondent sample: 
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Table 1 



Access Object 


Object Name & Columns 


Description 


Table 


a// alums 

• ID/ssn (J^) 

• name 

• degree 

• college 

• major 

• concentration 

• 2"^ major 

• year 

• rank 

• gpa 


Import listing of all 1982 - present 
undergraduate degree recipients from SIS+ 
located on the University’s mainframe. 

Data Source: MVS dataset 


Table 


uar alums 

• ID/ssn (^) 

• name 


Import listing of Undergraduate Research 
participants from Office of Undergraduate 
Research. 




• 

• 

• 


major 

year 

ugr_alum (1=ugr) 


Data Source: Excel table 


Query 
Type: make 
table 


combine ugr with all alums 


Creates [ugr&alum_combined]\ab\e\ Merges 
[AILalums] and [ugr_alums] tables by ID/ssn for 
manual matching process. 


Table 


uar&alums combined 

• ID/ssn (jP) 

• name 

• all_alums_year 

• college 

• all_alums_major 

• concentration 

• gpa 

• rank 

• ugr_alums_year 

• ugr_alums_major 

• ugr_alum 

• sort_key 


Manually update [ugr&alum_combined] tab\e 
per matching criteria. Table is sorted by 
[ugr&alum_combined].[sort-l<ey] field . 

fuar&alum combinedUuar aluml: 

‘1 ’ = Undergraduate Research Alumni 
‘2’= Matched Alumni (manual update) 

‘ ‘ = Not selected for survey 

Data Source: Make Table Query 


Query 
Type: make 
table 


ugr and alum selection 


Creates [matched_alumni] tab\e\ Select 'Ts and 
'2's from [ugr&alums_combined] tab\e. 


Table 


matched alumni 

• ID/ssn (J?) 

• name 

• ugr_alum 

• all_alums_year 

• college 

• all_alums_major 

• concentration 

• gpa 


Final listing of survey respondents. Includes 
856 undergraduate research (‘1 ’) alumni and 
1 ,675 non-research alumni (‘2’). Use to link to 
[address] f\\e. 

Data Source: Make Table Query 
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MAILING LABELS AND MANAGEMENT REPORTS 



Creating labels and reports within Access is simplified with the use of the report 
wizards. Table 2 lists the steps and fields used in Access to produce mailing labels and 
various management reports; 

Table 2 



Access Object 


Object Name & Columns 


Description 


Query 

Type: Select 


ID listing for Alumni office 


Select ID and name from [matched_alumni] and 
save as an Excel file 

[ID_listing_for_alum_office] to export to the 
Alumni Office in an e-mail attachment. 


Table 


address 

• ID/ssn (jP) 

• name 

• street! 

• street2 

• city 

• state 

• zip 


The Alumni Office provided an Excel 
spreadsheet listing current addresses 
based on the file sent, excluding those 
that are deceased or do not wish to be 
contacted (n=87). 

Data Source: Excel table 


Query 

Type; Select 
Report 


Matched Alumni 
Matched Alumni 


Create an inner join between [address] and 
[ugr&alum_combined] tab\es by /D/ssn for the 
Matched Alumni Report. The report lists the 
results of the matching process by year and 
major. Includes average GPA for each major 
within the each year. 

Data Source; Select Query 


Query 

Type; Select 
Report 


Matched Alumni 
Participants List 


Use the same query as for the matching 
report. Lists all alumni participants who 
receive a survey. Includes alumni 
group, graduation year, and major. 

Data Source: Select Query 
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Query 

Type: Select 



Report 
Type: Labels 



Report 
Type: Labels 



Labels - First Mailing 



Inside Label - First Mailing 



Outside Label - First Mailing 



Create outer joins between [matched_alumni] 
and [address] and between 
[matched_alumni]and [majors] tab\es for inside 
and outside mailing labels. 

Labels are produced using the Trim function. 
Trim displays the value of the address control , 
removing any leading or trailing spaces. 
(n=2444) 

• Inside label includes: 

• ID 

• Name 

• Graduation Year 

• Major (de-coded) 

• UGR-Alum code {‘T or ‘2’) 

•Outside label includes; 

• Name 

• Street 1 

• Street 2 

• City 

• State 
•Zip code 

Data Source: Select Query 



FOLLOW-UP AND SECOND MAILING 

As the surveys were returned, we kept track of the completed surveys as well as 
surveys returned due to incorrect addresses. Because our office staff in not networked for 
file sharing, these lists were maintained in Excel spreadsheets that were linked to the 
Access database. With only a 23% (542/2400) return rate on the initial survey, 
preparations for a second mailing began. A second copy of the survey was mailed to all 
non-respondents, excluding those for which we had an incorrect address. 

The table below lists the steps and fields used in Access to produce mailing labels 
for the second mailing: 



54 

o 

ERIC 



50 



Table 3 



Access Object 


Object Name & Columns 


Description 


Table 




Create link to Excel file [Completed surveys - 


Type: Link Table 


Completed surveys - first 


first mailing] to exclude names from the second 




mailing 

• ID/ssn (J^) 


mailing list. 


Table 




Create link to Excel file [Incorrect 


Type: Link Table 


Incorrect addresses 




• ID/ssn (^) 


addresses] to exclude names from the 
second mailing list. Also exported the 
list of names to the Alumni Office in an 
e-mail attachment for them to update 
their records. 

Data Source: Excel table 


Query 


Labels - Second Mailing 


Join [Labels-First Mailing] query with [completed 


Type: Select 




surveys - first mailing] and [incorrect addresses] 
linked tables by ID/ssn for inside and outside 


Report 


Labels - Second Mailing 


mailing labels. To exclude the names from the 


Type: Labels 




linked files, the query uses ‘Is Null’ criteria for 
both [completed surveys - first mailing]![ID/ssn] 
and [incorrect addresses]![ID/ssn] tie\ds. 

• Labels are produced using the same 
method used for the first mailing. 

For consistency and as a short cut, 
use the reports from the first mailing 
to create the second mailing reports. 
First change the name of the control 
source on the report properties then 
use the Save As option with the new 
names. 

Data Source: Select Query 



A total of 1,858 surveys were mailed in the follow-up. To date, 1,015 completed surveys 
were returned, for a total return rate of 45%. 
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OBSTACLES 



As with any project, below is our list of ‘obstacles encountered’ that challenged 
us as we progressed through the project. Many of these obstacles caused time delays in 
mailing the surveys. 

• Missing key field 

To get the most current addresses, we sent the Alumni Office separate lists of 
ID/ssn numbers; one of all the research alumni and one of only the non-research 
alumni. The initial address files returned from the Alumni Office did not include 
the ID/ssn field that was needed to link the address files to the other tables within 
Access. To rectify, we asked the Alumni Office to recreate the files and include 
the ID/ssn field. 

• Incorrect field format 

Our second request for the data from the Alumni Office included the ID/ssn field, 
however the field was formatted as a numeric field rather than text. This required 
an additional step of manually typing the leading zeros in student IDs where 
needed. 

• Records incorrectly sorted for labels 

We ran into a problem with sorting the records for the inside and outside labels. 
The two sets of labels were to be sorted in the same order to simplify the process 
of collating the surveys. To adhere to bulk mailing policies, the surveys had to be 
in zip code order. Within Access, the Zip code field was re-coded to a 5 digit zip 
code for the outside labels, creating a second zip code field. As a result, the 
inside and outside labels were inadvertently sorted using different keys. We 
could have reprinted the labels, but chose to sort them manually. This was a 
tedious and time-consuming process. 

The time delays became a problem because the surveys had already been dated and 
printed. Next time, we will ensure that the labels are ready to go prior to printing. 
Despite the obstacles, we are pleased with the outcomes of this project. The use of MS 
Access greatly aided our matching procedure, keeping track of respondents and non- 
respondents, and acquisition of mailing labels. 
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THE RELATIONSHIP BETWEEN STUDENT SUCCESS IN COLLEGE AND 
ASSESSMENT FOR REMEDIAL ASSISTANCE 



Michael J. Keller, Director of Policy Analysis and Research, 
Maryland Higher Education Commission 

Monica Williams-Randall, Policy Analyst and Research Specialist 
Maryland Higher Education Commission 



INTRODUCTION^ 

National and state studies show that many students continue to enter college unprepared 
for college-level work. According to the U.S. Department of Education, 78 percent of higher 
education institutions that enrolled college freshmen in 1 995 offered at least one remedial 
reading, writing, or mathematics course. In Maryland, all community colleges offer some form 
of remedial instruction. Ten of the State’s 13 public four-year campuses also offer remedial 
programs, although two provide it in math only. A study conducted by the Maryland Higher 
Education Commission (May 1 996) revealed that 47 percent of all new students at Mainland 
public campuses who enrolled directly from high school needed some form of remediation. 

Although the level of remediation is disconcerting, the problem of academically 
underprepared students is not a recent phenomenon. In 1 828, an article in the Yale Report 
admonished the college for admitting students with “defective preparation” (Maryland Higher 
Education Commission, 1996). In 1907, more than half of the students enrolling at Harvard, 
Princeton, and Columbia failed to meet entrance requirements and these schools established 
remedial programs (Payne and Lyman, 1996). By the mid part of the 1900s, most colleges and 
universities had established college preparatory programs to assist academically underprepared 
students (Mainland Higher Education Commission, 1996). Today, 90 percent of all public 
colleges and universities offer at least one remedial course (National Center for Education 
Statistics, 1998). 

Despite its history on college campuses, remedial education remains a controversial 
component of American higher education. Many academicians, trustees, legislators and average 
citizens question the need for providing courses on material that should have been covered in 
secondaiy school. Furthermore, the effectiveness of remedial programs is of particular concern 
to policy makers, educational administrators and faculty. 

However, most states do not know whether college-level remedial programs have been 
successful. A few have collected information about the completion rates of students in remedial 
classes, the subsequent performance of remedial students in regular college-level classes, and the 
short-term retention rates of remedial students. But there has been only limited investigation of 
the long-term success patterns of these students. The purpose of this paper is to explore these 
questions: What are the success rates (graduation and transfer) of students assessed for 
remediation at Mainland community colleges? What are the graduation rates of students 
assessed for remediation at Maryland public four-year campuses? Are there differences between 



Tables referenced in this article can be obtained by contacting the authors. 



the success rates of students based on the amount of remediation for which they were assessed? 
How do these rates compare to those of students who did not require remediation? 

PREVIOUS STUDIES 

Retention is problematic among students who are academically underprepared (Umoh, 
Eddy, and Spaulding, 1994; Schoenecker, Bollman and Evens, 1998). Studies that have been 
done on the effectiveness of remedial programs on retention generally support the view that 
efforts to improve the basic skills of underprepared students can produce meaningful outcomes. 

Both statewide and institutional assessments of remedial programs in Minnesota, New 
Jersey, Texas and Maryland suggest that remedial education has had a positive impact on short- 
term retention (Boylan and Bonham, 1992; Schoenecker, Bollman, and Evens, 1998; Tedrow 
and Rust, 1994; Bickford, Clagett, James, and Taibi, 1998). These studies contend that 
students who completed remediation requirements perform in college at levels generally 
comparable to those who did not require remediation. 

In addition, there has also been some research on the relationship between the extent of 
remediation and success in college. In Texas, researchers examined the extent that remedial 
programs prepared skill deficient students for success in college (England, 1994). Tracking the 
progress of first-time, full-time degree seeking students who entered between the fall of 1 99 1 
and the spring of 1 994, researchers found that students with skill deficiencies in two or three 
areas had the lowest levels of success. A study at Prince George’s Community College in 
Maryland also found that the achievement level of students varies by skill deficiency. Students 
lacking basic skills in two or more areas were less likely to persist, graduate or transfer to a four- 
year institution (Bickford, Clagett, James, and Taibi, 1998). The authors concluded that 
students prepared for college are more likely to graduate, transfer or persist from their 
freshman to sophomore year than students that required remedial assistance. 

METHOD 

The analysis in this paper was performed by matching unit record data elements from 
the Maryland Higher Education Commission’s Enrollment and Degree Information Systems 
(EIS and DIS) with those from its High School Graduate System (HGS). HGS collects 
information about the academic performance of graduates of Maryland high schools in the 
first year of collegiate study. Included are items dealing with assessment for remediation in 
math, English and reading. “Remedial” is defined in HGS as a course or series of courses or 
services designed to remedy deficiencies in preparation for college level work. 

EIS and DIS contain specified data elements for each student enrolled at a Maryland 
college or university and for each student who graduated from such a campus. Each record 
for the public institutions is identified by a student’s social security number, making it 
possible to examine student persistence, graduation and transfer patterns by matching records 
among years. 

There are several limitations inherent in this study: 

1 . While HGS collects annualized information (students who enrolled in the summer, fall 
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and spring), EIS consists of a snapshot of students in attendance at a point of time each 
fall. For this reason, it is impossible to match the records of EIS with students in HGS 
who enrolled in other than the fall semester of any particular year. Hence, only students 
who entered college in the fall are included in this analysis. 

2. HGS includes data only about students who entered college directly from high school. 
Therefore, this study contains no information about remedial assistance provided to other 
new full-time freshmen. 

3. HGS is in its sixth reporting cycle, with 1991-1992 being the earliest year. Therefore, 
long-term retention and graduation tracking is available for just a few cohorts. Analyses 
will be performed for additional years as more information is collected. 

4. Campus policies regarding remediation and assessment in Maryland vary. Equally 
underprepared students may be required to take different types of remediation, depending 
on the institution at which they enroll. HGS collects information about the number of 
students who were assessed as needing remediation in certain areas, and institutions 
diverge in terms of whether these individuals actually have to enroll in such classes. In 
addition, the availability of remedial information differs among the State’s colleges and 
universities. As a result, the definition of “remedial” used in this study is not apt to be 
applied consistently among institutions at this time. However, this is expected to change 
in the near future at the State’s community colleges. The instructional deans at these 
institutions have undertaken a project that envisions the implementation of statewide 
instruments, standards and policies for testing and placing remedial students within two 
years. 

DATA ANALYSIS 

New full-time freshmen in the entering classes of 1991, 1992 and 1993 at Maryland 
community colleges and those matriculating at the State’s public four-year campuses in 1991 
and 1 992 were organized into four categories reflecting the amount of remedial assistance for 
which they were assessed in college: 1) those who were assessed but needed no remediation, 
and those who required remediation in 2) either math, English or reading, 3) in two of these 
areas, or 4) in all of these areas. Using this classification, the following analyses were 
conducted: 

• The percentage of corrununity college students who transferred to a public four-year 
campus in Maryland and/or earned a certificate or associate degree from any two-year 
institution in the State within four years of matriculation. 

• The percentage of students at public four-year institutions who earned a bachelor’s 
degree from any public campus in the State within five or six years after entry. 



Community Colleges 



The community college information is particularly pertinent, since the greatest 
amount of remedial education in Maryland higher education, as elsewhere, is performed at 
two-year institutions. A substantial number of community college students who entered 
directly from high school required help in basic skills (table 1 ). In each year, more than 60 
percent of the two-year students needed some kind of remedial assistance, and more than 30 
percent required it in two areas or more. 

The statistics in table 2 show that, for each of the three cohorts, success (as defined 
by the percentage of students transferring or graduating within four years) declined as the 
number of areas in which remediation was needed increased. Adding the percentage of 
students who remained enrolled at a community college to this “success rate” did not change 
this pattern. The figures were very consistent across years. With very few exceptions, this 
trend was not affected by gender and ethnicity (table 3). 

Public Four-Year Campuses 

Far fewer smdents at Maryland’s public four-year colleges and universities required 
remedial assistance. More than 80 percent of the new full-time freshmen in 1991 and 1992 
were assessed as not needing remediation (table 4). 

The findings from the four-year institutions resembled those at the community 
colleges (table 5). Among smdents in the class of 1991, those who required no remediation 
had the highest graduation rate after both five and six years. Students who had been assessed 
for remedial help in just one area were next highest, while those who needed assistance in 
two or more areas experienced the lowest graduation rates. When the percentage of smdents 
who remained enrolled at their college is added to the graduation rate, students who required 
help in all three areas have the lowest level of success. The distinction between the remedial 
categories is even more apparent when one examines the graduation and retention rate of 
entering smdents in 1992 five years after matriculation. As with community college smdents, 
these findings were consistent in a majority of cases across gender and ethnicity (table 6). 

POLICY ISSUES EMERGING FROM THE STUDY 

This smdy documents that recent high school graduates who are academically 
underprepared are less likely to be successful in college. Furthermore, there is a relationship 
between the extent of remediation and college success. Smdents who lack the basic skills in two 
or more academic areas are less likely to persist or graduate. The following policy issues arise 
from this smdy. 

Should students who are assessed as needing remediation be required to take it 
regardless of their course of study? 

One of the limitations of this smdy is that it is unable to distinguish between the 
performance of smdents who were assessed for remediation and took it and those who were 
assessed but did not. However, research suggests that there is a relationship between successful 
completion of recommended remedial coursework and overall success in college and that 
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remedial programs can have a positive impact on underprepared students. Students who enter 
college lacking the basic skills needed to perform effectively are already disadvantaged 
scholastically. These individuals almost certainly will reduce further their chances of success if 
they do not take remedial coursework in their areas of deficiency. For this reason, it makes 
sense for college administrators, particularly at open admissions institutions, to require entering 
students who need remedial assistance to take it as a prerequisite for enrolling in regular courses. 
To do otherwise would be to provide access to some students with little hope that they will be 
successful. 

Should students who plan to enter postsecondary education at any level he 
encouraged or even required to take a college preparatory curriculum? 

The major finding of this study is that students who enter college needing remediation 
are less likely to graduate or transfer than those who do not— and that the chances of success 
decline as the number of areas in which remediation is required increase. The message is clear. 
Students who enter college underprepared are already facing an uphill struggle, regardless of 
their intellect, ambition or determination. And while colleges and universities may be able to fix 
a student’s shortcomings in one field with a one- or two-semester class, the problem becomes 
more severe for students whose academic deficiencies range into multiple areas. 

These problems can be largely remedied by the choices that students make in high 
school. Research by the Southern Regional Education Board and several states have found that 
rigorous academic preparation in secondary school, and the seriousness with which students 
apply themselves to their high school studies, affect the level of remediation they need in 
college. The intensity of curriculum makes a difference. Therefore, efforts should be 
undertaken to keep secondary school students and their parents informed about the importance 
of following a recommended course of study in preparing for college and to stress the value of 
developing and applying good study skills. Indeed, a number of states that have implemented 
policies that require or encourage students to complete a more challenging college preparatory 
curriculum have reported a decline in the number who need remedial coursework. In short, 
policy makers interested in shrinking the remedial empire in higher education need to focus 
attention on what is happening in the K-12 arena. 

Although there is evidence that well-selected courses make a difference, there are 
students who complete a college preparatory program in high school and are still assessed for 
remediation in college. This anomaly may be due to the variation in the content of courses taken 
in specific subject areas. Furthermore, the quality of college preparatory programs differ from 
school to school and how well students do in college courses are also factors. 

Should colleges and universities establish special programs or requirements for students 
who require extensive amounts of remediation? 

The results of this study suggest this is essential if these students are to have any 
chance of success at all. Accommodating students who show a general lack of preparation 
for college work requires a more comprehensive set of learning support services and a . 
supportive learning environment (Noel, 1987). Many colleges and universities have 
developed innovative approaches to improve the success rates of underprepared students. 

The key attributes of these programs are their ability to integrate students into the campus 
environment by identifying and placing underprepared students into remedial courses. 
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providing small classes and personalized instruction, and collaborating with other campus 
support services. Research suggests that colleges able to provide multiple support services 
can enhance the overall achievement of underprepared students. 

What kinds of early intervention programs should be established to reduce the 
number of recent high school graduates who enter college needing remediation in math, 
English, and reading? 

To raise the level of academic preparedness, early intervention programs must be 
established and target students as early as middle school. These programs should be designed 
to educate students and parents about college admissions requirements before students enter 
high school. Early intervention programs should help them understand the consequences of 
their decisions and should also encourage students to seriously consider their future choices. 
Furthermore, middle school students should be encouraged to enroll in a rigorous curriculum in 
high school that consists of core courses in science, mathematics, English and social studies. 
These programs should increase the academic ability of students regardless of whether or not 
they continue their education after high school. 

Should colleges and secondary school systems partner to create parental education 
programs to provide an understanding of the consequences for students who come to 
college poorly prepared? 

Many parents who have not been to college themselves are often intimidated by schools, and 
they may have an inadequate understanding of the consequences for students who come to 
college underprepared. Establishing a partnership between college faculty, secondary school 
teachers and administrators, and parents could help to enhance understanding of the importance 
of enrolling in a rigorous curriculum in high school as well as increase the amount of out-of- 
school time that students apply to academic endeavors. 
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A FORMULA FOR WRITING ENVIRONMENTAL SCANS FOR 
COMMUNITY COLLEGES 

Marcia M. Lee, Ph.D. 

Director, Office of Institutional Research and Planning 
Westchester Community College 



Introduction 

Community Colleges have a mission substantially more focused than four-year 
institutions, specifically to serve the local community in providing post secondary 
education and workforce training. To do this effectively it is necessary for administrators 
and faculty to keep current on various aspects of the community they serve. Many 
Offices of Institutional Research, however, shy away from conducting environmental 
scans because the task seems daunting or the time too limited. Neither needs to be the 
case. 

This paper proposes a five-part formula for conducting an environmental scan and 
writing up the results. It is designed to make the project as easy as possible, focused, and 
usable the next time around (once every three years, if possible). Besides making the 
environmental scan doable, the objective of this formula is to provide the data necessary 
for college faculty and administrators to have a good sense of future student enrollments, 
and be able to make informed decisions for planning future academic programs, students 
services, off-campus locations, and other long range objectives. 



Part I: Focus the Subject Matter on Five Topics 

There are an endless number of topics about the community or county one could 
address in an environmental scan, but the following seem to be the most important for 
decision making: 

1. Population Trends: The first order of business is an analysis of 

the absolute growth or decline of the county population followed by an analysis 
of any geographic shifts of the population within the county. Is the population 
moving out of the cities toward more rural areas; is it moving north or south; is it 
remaining fairly stable in location? Next, a breakdown of the ethnic background 
of the population with emphasis on trends. What ethnic groups are growing the 
fastest? Where? Third, the age trends. Is the population getting older; is the 
birthrate increasing? Is there a bulge in the 34 to 50 aged population? Last, the 
gender. Do men and women comprise roughly the same percentage of the 
population at various ages, or are there marked differences? 



Economic Outlook: A review of the overall economy of the 
county is the first focus here. What are the prospects for economic growth in 
the next five years in terms of annual percentage growth rate? Is the county 
coming out of or going into a recession? Next should come an analysis of the 
sectors within the economy indicating which ones are growing. The standard 
sectors used in most government reports include: (1) 'Wholesale & Retail 
Trade, (2) Services, (3) Government, (4) Manufacturing, (5) Finance, 
Insurance & Real Estate (F.I.R.E), (6) Transportation, and (7) Construction. 
Included in Services are two particularly important sub-sectors— the health care 
industry and the information technology industry — two probable areas of 
growth, depending on where you live. 

Job Openings: Projections showing the number of job openings in 
the next five to ten years is the focus of this section. Select those occupations 
requiring post-secondary education. The state or county Department of Labor 
should have the data broken down by the level of higher education required, 
including. Occupations Requiring Some Post-Secondary Education, 
Occupations Requiring an Associate ’s Degree, and Occupations Requiring a 
Bachelor’s Degree. 

High School Graduates: Recent high school graduates in the 
county comprise an age group particularly important to community colleges 
because they tend to be the largest source of first-time, full-time students. The 
trend over a period of years is important. Currently, an annual increase in the 
number of high school graduates is occurring throughout the country as the 
baby boomlet (children of baby boomers graduating from high school) plays 
out. This trend should continue until 2008. A comparison of the number of 
high school graduates per year to the percent entering the community college 
the following fall semester is a useful bit of information, too. 

Commutation and Transportation: Community colleges also are 
unique in the degree to which most of the students use some form of vehicular 
transportation each day. Major road improvements, changed bus routes, and 
unreliable commuter train schedules can have an impact on college 
enrollments. This section of the environmental scan should list the major road 
repairs for the next three years and their completion dates, new or revised bus 
routes, and chronic commuter rail problems. Of course, adequate parking is a 
factor not to be overlooked, even if, technically, it is located on campus. 



II: Collection of Data 



Never has the old adage, A job well begun is a job half done, been more applicable 
here. Begin with the County Planning Department, and do not just communicate by 
phone or E-mail. Make an appointment to meet with the top planner that will see you. 
Take any pertinent handouts and reports that Planning Departments tend to display in 
racks and open files, and ask to see their maps. The Planning Department is particularly 
good for population trends and transportation maps. If the county also has an Office of 
Economic Development, go there, too. This Office usually publishes booklets for 
perspective companies that are thinking of locating in the county. The booklet will 
probably include a wealth of information on the local economy. 

For job opening trends in the county, the state Department of Labor is probably the 
best source. Ask for the chief analyst, who should be able to provide projections for the 
number of job opening per occupation in your region, if not county, further broken down 
by the amount of education required. Ask only for the jobs requiring (1) some higher 
education, (2) an associates degrees, and (3) a bachelor’s degrees. A separate chart of the 
number of projected openings for the top ten job categories for each level of higher 
education is an effective way to present this data. 

For information on high school graduates contact either the local association of 
schools in your county or the state Department of Education. Your own admissions 
director is a good source for the number of June graduates who enroll the following fall. 

For information on transportation developments, the county Transportation 
Department may be able to add to the information available from the Planning 
Department, or, if nothing is available, the State Department of Transportation may be 
helpful. If you are located in a large metropolitan region, there may also be a regional 
organization. 

The most important objective is to develop a working relationship with these planners 
and officials. To the extent you can quickly call on these people for information in the 
future, your value to the College increases considerably. It is worth the time to develop 
personal contacts. 
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Part III: Use Tables and Charts and Maps 



Each major point should be supported either with a map, a table, or a chart. For 
population shifts within the county and transportation problems, maps are particularly 
useful. For demographic trends and job sector trends, bar charts and pie charts are very 
effective. For trends in high school graduation rates, and job opening trends, tables are 
useful. In fact, tables are useful to back up most types of bar charts and pie charts. 

Charts, maps, and tables not only provide a quick visual way to present data, but they 
break up the page and give a nice presentation that can make the difference between 
someone willing to wade through the report or not. 

Part IV: Develop a Detailed, Concise Table of Contents 

Having spent all the time to collect and analyze the data, it is well worth the 
investment of time to develop a detailed and concise Table of Contents. It allows the 
perspective reader to see quickly where information is located, and immediately displays 
the organizational layout of the report. Making it a part of the front page, such as the 
Reader’s Digest, also can be effective. 

Part V: Include an Executive Summary and Conclusion 

Some researchers think that by not including an Executive Summary, the reader will 
be forced to read the entire report. Wrong. More than likely, the reader will not read the 
report at all. One of the greatest challenges faced today by those in Institutional Research 
is to get the rest of the College to actually read the reports produced. 

Executive summaries are a large part of the answer. Everyone likes to feel they have 
cut through the superfluous and acquired the nuts and bolts of the report by reading just a 
few pages. The secret for Institutional Researchers, therefore, is to have the Executive 
Summary cover everything in shorthand form. A modified outline format lends itself to 
this task. Begin the summary with a brief statement of what the report covers, i.e., this 
environmental scan focuses on five factors (1) Population Trends, (2) the Economy, (3) 
Job Market Trends, (4) High School Graduation Trends, and (5) Commutation Trends. 

At the end of each factor that is summarized, it may be appropriate to draw a 
conclusion or make a recommendation. If so, these statements might best be put in 
italics, and even bolded to draw attention to them. Too often Institutional Researchers 
shy away from offering conclusions or making recommendations. Yet, they are the ones 
that have been working with the data the most. It is time for more Offices of Institutional 
Research to provide this higher level of service. 
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Conclusion 



Environmental Scans are not difficult to produce, once a specific format is developed 
and the best sources for the data are identified. Since these scans should be conducted 
periodically (every three years, if possible), it is well worth the effort, initially, to 
establish close personal working relationships with planners at the county and state level. 

Writing the report is only half the challenge. The other half is to make sure that 
faculty and administrators read the report and incorporate the information into their 
decision making in developing new curricula, changes in student services, locations for 
off-campus sites and other long range plans. In accomplishing this later objective, the 
importance of maps, charts and graphs to fortify the data should not be underestimated, as 
well as the effectiveness of a concise and complete Table of Contents and an well-written 
Executive Summary. As an example, an executive summary for an environmental scan 
conducted for Westchester Community College follows in Appendix I.* 



* An example of a complete environmental scan report using this format is available on 
request. It is entitled: Sources of Student Enrollments for the Next Decade, A 
Demographic and Economic Outlook: 1998 to 2008, by Marcia M. Lee, Ph.D., Office 
of Institutional Research and Planning, Westchester Community College, Valhalla, New 
York June 1998. 
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Appendix I 



An Executive Summary Example 

Sources of Student Enrollments for the Next Decade 
A Demographic and Economic Outlook: 1998-2008 



With the population of Westchester county projected to grow only minimally in the next 
decade, Westchester Community College will have to look within the population and 
outside the county to identify potential new students. 

This report focuses on five factors, (1) Population Trends, (2) the Economic Outlook (3) 
Job Market Trends, (4) High School Graduation Trends, and (5) Commutation Trends, to 
analyze their impact on future student enrollments and to suggest groups within the 
population the College might seek to attract. 



Population Trends : Four major population shifts are projected to occur within the county 
and in the greater Hudson Vdley Region in the coming decade (1998 to 2008). 

1. The rapid growth in population in northern Westchester and southern Putnam 
county is not expected to continue due to the recent implementation of the New 
York City 1997 watershed regulations that will constrain housing production 
considerably. The College should not look to northern Westchester or 
Putnam county in the next decade for a rapidly growing population to supply 
a new source of additional students. 

2. Fortunately, southern Westchester, after decades of population decline is 
expecting a modest growth caused by in-migration from the Bronx ^d other 
metropolitan areas. In terms of potential sources of new enrollment the 
southern section of the county, particularly Yonkers, Mount Vernon, and the 
northern Bronx are places to look. 

3. The Hispanic population will undergo the largest growth of any ethnic group 
between the years 2000 and 2010. It is projected to increase by one-third 
(32.4%) adding 60,667 to the population. The potential new growth in the 
student body will come primarily from students of Hispanic background. 

4. The population of Westchester county will continue to grow older, such that by 
2010 almost one-fifth (17.1%) will be 65 and older and less than one fifth 
(17.1%) will be 19 and under. In developing new programs the College should 
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give special consideration to courses and programs that appeal to an older 
generation of people. 



High School Graduates : In June, 1997 the baby boomlet (children of the baby boomers 
graduating from high school) began to kick in and the number of high school graduates is 
expected to gradually increase until 2008. High School graduates, therefore, are a 
growing pool of potential students for the next ten years. 

Economic Outlook : The economic outlook for Westchester and the Hudson Valley 
Region is favorable with an annual growth rate projected at between 1.2% to 2.2% for at 
least the next five years. Neighboring counties are expected to have an even higher 
growth rate with Fairfield, Connecticut projected to grow at an annual rate of 2.6% and 
northern New Jersey at 2.5%. Since many WCC students find employment in these 
areas outside the county, this should be kept in mind when calculating the job market 
for a prospective degree or certificate program. 

Job Market Outlook : In the Hudson Valley Region the shift to a service economy will 
continue, going from 32.9% of the economy in 1996 to 37.3% by 2005. Within this 
sector the largest number of jobs will occur in the health services industry with 18,400 
new jobs projected. The fastest growing area, however, is the business services sector, 
particularly information systems, with a 52% rate if increase projected, adding 17,200 
new jobs by 2005. All the other sectors will see little or no increase including the 
Government sector and the Finance, Insurance and Real Estate (F.I.R.E.) sector. In 
planning new curricula, therefore, the health sector and the information systems sector 
are expected to far out-distance the others in future job openings in the next decade. 

Jobs Requiring Higher Education : The top ten occupations with the largest number of 
projected job openings that require some post secondary education, an Associate’s Degree 
or a Bachelor’s degree are: 



Occupation 



Annual Job Openings 



1 . Secretaries (excluding legal and medical) 

2. Teachers, Secondary school 

3. Registered Nurses 

4. Teachers, Elementary 

5. Automotive Mechanics 

6. Accountants and Auditors 

7. Computer Programmers 

8. Data Entry-Insurance 

9. Social Workers, (Except Med. & Psych.) 

1 0. Hairdressers and Cosmetologists 



.) 700 

650 



3,790 

1,610 

890 

830 

820 

770 

730 

710 
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Only one of the top ten occupation requires an Associate’s degree (registered nurses). 
Moreover, only 29,700 job openings requiring Associate’s degree are projected in the 
next decade, compared to 1 19,790 requiring a Bachelor’s degree and 57,220 requiring 
some post secondary education. 

Commutation : Surveys conducted on WCC students confirm that the commuting time to 
and from college is an important factor in attending the College, especially for students 
thirty and over. 

Improvements on the lower Sprain Brook Parkway that are near completion should 
shorten the commuting time from the Bronx, southern Yonkers, and Mount Vernon. 
Coupled with projected population increases in the southern section of the county, and 
an increase in the number of Hispanics reaching college age, these commutation 
improvements provide an additional reason to target new students in the Bronx, 
southern Yonkers and Mount Vernon. 

Expansion of the Taconic Parkway from four to six lanes in Yorktown (scheduled to 
begin in 2000) will lengthen the commuting time for students in northern Westchester to 
the Peekskill and Valhalla campuses until completion. Improvements on lOOC at the 
entrance to WCC down to Virigina Road will also cause delays. Measures to indicate 
the temporary nature of these construction projects and to help abate their 
inconvenience are recommended. 

Where is Valhalla? This question will be explored in a countywide survey to be 
conducted next fall, but informal questioning indicates that the College’s convenient and 
central location is not fully working for it. 

To be sure, other factors may influence future enrollments at Westchester Community 
College, such as the development of distance learning, increased demand for remedial 
education, and more competition from four-year colleges for traditional WCC students. 
The demographic and economic trends discussed in the report, however, are sure to have 
an important impact, too. 
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FINANCING EDUCATION 

An Analysis from the Results of the Survey of Graduates 1989-1997 
Qing Lin Mack and Jennifer A. Brown 
Academic Affairs and Research 
Connecticut State University System 



Introduction 



Public debate about the cost and price of higher education has become more 
passionate in recent years. National data shows that there have been significant increases 
in tuition and fees, articles appear regularly in newspapers about rapidly increasing 
tuition prices and legislatures all over the country are asking why universities and 
colleges charge what they charge. 

Within our Council of Presidents and Board of Trustees, like those of many public 
institutions, there has been much discussion and debate about tuition and fee policies and 
about financial aid policies in a public, four-year comprehensive university with a 
mission of providing access to higher education. One recent policy debate has been over 
the provision of non-need based aid. The data policy makers have examined to date has 
been the somewhat gross measure of ‘unmet need’ estimates. 

Our goal in this paper is to examine a rich set of data gathered over the past nine 
years (1989 through 1997) from our graduating seniors and to take a look at student 
reports on: 

• how they have paid for their education, 

• what forms of financial aid they have received, 

• median amounts of student loans, and 

• students’ level of concern about student loan repayments 

In each case, we will be examining the patterns of stability and change in student 
respondents’ responses. 

Before turning to the analysis of payment sources and financial aid, we briefly 
review the economic background in Connecticut and the characteristics of the students 
who attend Central, Eastern, Southern and Western Connecticut State Universities. A 
description of the survey methodology and response rates can be found in appendix A. 

Background: State Context 

Connecticut, as much of the Northeast, went from times of relative plenty in the 
late I980’s into an economic recession during the early I990’s. Table 1 shows the 
increases in tuition and fees over that period of time at the four universities. The average 
of the four institutions’ tuition and fees for full time undergraduate are presented. It is 
clear that the cost of attendance increased dramatically between 1989 and 1997. The nine 





year increase was in fact 151%, a rate considerably higher than inflation. The 9 year 
increase on CPI (Consumer Price Index) was 28% and 30% for HEPI (Higher Education 
Price Index). 



Table 1, Tuition and Fees 











1989 


1510 


12.4 


1990 


1586 


5.0 


1991 


1950 


23.0 


1992 


2504 


28.4 


1993 


2927 


16.9 


1994 


3154 


7.8 


1995 


3350 


6.2 


1996 


3479 


3.9 


1997 


3785 


8.8 



It is true that the tuition and fee prices listed above, even in 1997, are bargain 
prices. In comparison to the prices of private institutions and to doctoral granting publics, 
they may look laughably small. We would argue, however, that they are not small 
amounts to our students. 



Background: CSU Student Characteristics 



The four universities of the CSU system enroll students typical of four year 
public, comprehensive institutions. Of the 33 thousand students enrolled just over one 
half are full time undergraduates, 25% part time undergraduates and the remaining 23% 
full and part time graduates. 

Among the full time undergraduates, most (87%) are traditional age and 13% are 
25 and older. Among part time undergraduates, almost two thirds are 25 and older. 

Most important as a backdrop to the analysis of student educational financing is 
the information that almost one half (47%) of CSU students responding to the Surveys of 
Graduates report that neither of their parents or guardians ever attended college. Only 
13% or so report that both parents or guardians have graduated from college. In a state 
which has a higher than average population of persons 25 and over who have completed 
college, this is a clear indicator that CSU students are members of families who do not 
typically fall into high socio-economic ranks. 

This is further reinforced by the consistent findings in our Current Student 
Surveys (1995-1997) that about 80% of the responding students reported they were 
employed while attending the university. Just over forty percent say that they are 
employed full-time and almost 40% that they are employed part-time. 



Turning to the analysis of student educational financing, we will first examine the 
student respondents answers to the question, ‘How did you pay for your education at 
CSU’? In the interests of clarity, we will present only three years’ of survey results in our 
analysis; 1989, 1993 and 1997. Data on the intervening years has been examined and is in 
keeping with the trends discussed. 



Pavine for Education 

In the survey, we asked our graduates ‘How did you pay for your education at 
CSU?’ The choices for the answer were: personal earnings or savings, parents’ 
(guardians’) support, spouse support, veterans benefits, employer reimbursed my tuition, 
scholarship from private sponsor, university scholarship, financial aid including loans, 
vocational rehabilitation or other agency support, and personal loans. Students were 
encouraged to choose all that apply. 



Table 2, Payment for Education 

(table presented using percentage) 

















Personal earnings 


63 


55 


55 


Parents support 


49 


56 


45 


Financial aid incl. loans 


29 


25 


39 


Employer reimbursement 


10 


10 


11 


Spouse support 


8 


6 


7 


Private scholarship 


7 


7 


9 


Veterans benefits 


4 


3 


5 


University scholarship 


4 


7 


8 


Voc. rehab or other agency 


— 


1 


1 


Personal loans 


— 


— 


9 



Note: * We began asking the question about vocational rehabilitation or other agency 

support after the 1989 survey and about personal loans in 1997. 

* The columns do not add to 100% as students could select more than one 
option. 



Table 2 shows that in general, personal earnings is the largest source of funding 
for CSU students. The percentage of graduates who checked personal earning decreased a 
little between 1989 (63%) to 1993 (55%) but has remained stable since that time. 
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Parental support is the second largest source that CSU graduates used in paying 
their education. The percentage of respondents who reported that they used parent^ 
support for their education showed no dramatic change in the last 9 years 

Financial aid was another major source for CSU students. In 1989, a 29% of the 
respondents reported that they had financial aid. This proportion went down a little 
during the 90’s, to 25% in 1993 but has since risen significantly to 39% in 1997. 

A more detailed analysis was done to see what combinations of financial sources 
students used to pay for their education. Graph 1 to Graph 3 shows the patterns shifting in 
the past 9 years for financial aid. 



Graph 1 



Top Eight Combinations of Financial Aid 
1989 



parents suppt. only ; . ■ 



p e rs o h a I ea rn . :6 n ly 



personal earn. & parents suppt. 



;pers6hal earn.r&^finar cial aid 



pers. earn.&.parq nts suppt.& fin. aid 



empV reimbursement 



'spouse suppt. 



finaricfel aid inci loans only 



In 1989, majority of the CSU graduates who responded to the survey said parents 
support or personal earnings was the only source for paying their education at CSU. 
Financial aid ranked on the bottom of the top eight combinations of payment method. 
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Graph 2 



Top Eight Combinations of Financial Aid 
1993 



parents suppt.-^^ f. . 



personal earn. &^ p suppt. 



persorialvearn.^pnlY v 



persona) earn. & financial aid 



pars. Barn. & parents suppt. & fin. aid 



f:fthdriclai aid inci loans only 



^emj). reimbursement 



peiil sonal earn, and emp. reim. 



In 1993, more CSU graduates reported that they were solid depended on financial 
aid for paying their education. Financial aid ranked as the 6'^ most important source in 
1993. (Graph 2) 

Graph 3 



Top Eight Combinations of Financial Aid 
1997 



^ p are nts^ s u ppt on ly 



■Kperspha 



, .<;|persohalceaT :. 
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suppt. & fin. aid 
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By 1997, financial aid as a solid funding source of education went up to be the 4* *'’ 
of the top eight important sources for paying education. (Graph 3) 

Overall, the biggest change in the student’s self reported sources of funding for 
education is the increasing role played by financial aid. The percentage reporting reliance 
on contributions made by students’ families and on their own earnings have remained 
relatively stable and at high levels. This suggests that the increases in tuition costs have 
not led to increase parental and personal sources of funding, but to increase reliance on 
financial aid. We therefore turned our attention to the types of financial aid received by 
the student respondents. 
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Types of Financial Aid 




Students were asked ‘What types of financial aid did you receive while attending CSU?’ 
The response choices included: pell grants, work study, student loans, other federal 
aid, state financial aid, tuition wavers or did not receive financial aid. Students were 
encouraged to choose all that apply. 

The data in Table 3 shows the percentage of students reporting having received one or 
more forms of financial aid. The columns do not add to 100% as students could select 
more than one option. 

The most frequently cited form of financial aid was student loans. While the 
percentage of graduates indicating this form of aid decreased from 30% in 1989 to 23% 
in 1993, it increased significantly between 1994 and 1997, reaching almost 40% of the 
students who reported financial aid types. 



Table 3, Types of Financial Aid Received 



(table presented using percentage) 





Student loan 


30 


23 


39 


Pell grants 


15 


16 


18 


Work study 


8 


6 


8 


State financial aid 


8 


8 


9 


Other federal aid 


4 


5 


8 


Tuition waivers 


— 


4 


7 


Didn’t receive aid 


62 


65 


48 



Students who received financial aid often received more than one form of aid. 
Comparing the combinations of financial aid for 1989, 1993 and 1997, ‘student loan 
only’ is still the most frequent form of aid. The following Table 4 shows the detailed data 
on the financial aid ‘packets’ for each of the three years. 
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Table 4, Financial Aid Combinations 











Student loan only 


269 


168 


228 


Student loan and pell grant 


100 


122 


81 


Student loan & one of the other forms (work 
study, other federal aid, other state aid, 
tuition wavers) 


191 


223 


196 


All other combinations except student loans 
(pell grant, work study, other federal aid, 
other state aid, tuition wavers) 


131 


182 


132 


Total Number Received Financial Aid 


691 


695 


637 


Total Number of Responses 


1897 


2239 


1285 


% Receive Financial Aid 


36% 


31% 


50% 


% Receive Student Loan only 


14% 


8% 


18% 



Table 4 shows that using the aggregated figures for those reporting receiving one 
or more forms of financial aid, the percentage of all the respondents reporting one or 
more forms of aid was just over one third in 1989 and just under one third in 1993. By 
1997, however, that percentage had increased to 50% of the total respondents. There were 
14% of the respondents said student loan was the only source of financial aid. This 
number decreased to 8% in 93 and back up to 18% in 1997. 

In fact, the current student body who depended on financial aid maybe greater 
than the figures we have from graduates who responded to the survey. In 1997, our 
finance office reported that 60% of full-time CSU students receive financial assistance, 
including loans. The loans made to students increased 75% from 1994 to 1997. In 1994, it 
was 23 million dollars and by 1997, 40 million dollars. 

This lead us to questions about loan amounts and concern about repayment. 



Loan Indebtedness 



The number of graduates who reported having some dollar amount of loan debts 
was smaller than the number of graduates who reported receiving loans in the previous 
question. This may be because graduates who are not up-to-date on payments do not want 
to report or just that some people do not like revealing financial data. 

Table 5 showed the three years data on loan amounts for all the graduates who 
returned the survey. 
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Table 5, Amount of Loan 



















Number 


Percent 


Number 


Percent 


Number 


Percent 


None 


1103 


58.1 


1476 


65.9 


637 


49.6 


$1 to $2,499 


238 


12.5 


190 


8.5 


49 


3.8 


$2,500 to $4,999 


214 


11.3 


186 


8.3 


65 


5.1 


$5,000 to $7,499 


137 


7.2 


137 


6.1 


104 


8.1 


$7,500 to $9,999 


88 


4.6 


90 


4.0 


67 


5.2 


$10,000 to $12,499 


48 


2.5 


47 


2.1 


109 


8.5 


$12,500 to $14,999 


17 


0.9 


18 


0.8 


76 


5.9 


$15,000 or more 


7 


0.4 


33 


1.5 


149 


11.6 


Total 


1852 


97.6 


2177 


97.2 


1256 


97.4 


Didn’t give an answer 


45 


2.4 


62 


2.8 


29 


2.3 


Total Responses 


1897 


100.0 


2239 


100.0 


1285 


100.0 



The median student loan indebtedness in 1989 was $4,100, and by 1993 was in 
$4,663. However, by 1997, the median student loan indebtedness was $10,573, a 158% 
increase from 1989. The change was so significant that we checked and rechecked the 
calculations, wondering if we had made a mistake. We do not think that we have. 
Students are not only relying increasingly on financial aid, but they are relying 
increasingly on loans and in increasing amounts. 

Are they concerned about load indebtedness? 



Concern about Loan repayment 

Among those respondents who reported some dollar amount of loan debt, we 
examined their responses to the question about concern with loan repayment. Response 
options included: no loans, no concern, some concern, major concern, and ‘expect that 
my family will pay’. Graph 4 shows that there has been a big change in our students’ 
responses between 1989 and 1997. 

The percent of the respondents who reported that they had no concern at all about 
repayment dropped from 67% in 1989 to 51% in 1993 and to 44% in 1997. The 
respondents who had some concern or major concern increased from 28% in 1989 to 40% 
in 1993 and 50% in 1997. The percentage of students who expect families to pay back 
their indebtedness is very small, less that one percent. 
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Graph 4 




Summary of findings 

• While our tuition and fees are comparatively low, they have increased significantly 
over the last 9 years. 

• The price we are charging for education is a struggle for our students and their 
families. The most significant sources of funding for education among our students 
continue to be their own earnings and their families. 

• There has been a dramatic increase in reliance on financial aid, especially in the last 
four years. 

• There has been a shift within financial aid sources to more student loans. 

• The median amount of loan indebtedness has increased dramatically in the last four 
years. 

• Students are increasingly concerned about their ability to repay education loans. 

Policy implications 

• Our students and their families are clearly contributing as much as they can from their 
own pockets to pay for their education. It appears that those sources have reached 
their limits and more students are relying on financial aid to fill the gap. What ought 
we to be doing in response to this? Holding tuition and fees steady? Requesting that 
the state resume a greater responsibility for the costs of instruction? Pouring more 
money into financial aid? 

• National policy decisions have put more emphasis on loans than grants in recent years 
and our students are increasingly feeling the burden of this shift. Are there ways in 
which we could reduce the reliance on student loans for college costs, particularly 
among our less wealthy students? 

• Students are expressing heightened degrees of concern about debt. Can we provide 



services which help them to plan debt management? Are there loan forgiveness 
programs we could develop targeted at sectors of the Connecticut economy in which 
employers are currently having difficulty hiring? 

• Are we doing enough within university resources to provide financial resources to 
students? Do we need to plan and implement the development of significant 
scholarship funds? 

• How does the increasing reliance on financial aid square with our mission of access? 
Is the need to borrow reducing the number of students and families able to attend 
college? 

We hope that our analysis provides useful information for our policy makers as they 

struggle with these questions. 



Appendix A 

I. Survey Back Ground: 

In 1989, the Connecticut State University system initiated an annual survey of 
graduating seniors and which has been collected every year since that time. Typically, the 
surveys are mailed out in late fall/early winter, followed up with a postcard reminder and 
finally, three weeks later, another complete survey. The data are used at the campus and 
the system level for analysis of the post-graduation activities of our graduates as well as 
analysis of their undergraduate experience with programs and services at CSU. The 
survey instrument also includes questions on students’ employment during their 
undergraduate careers, their image of their campus and their strategies for financing their 
education. 

II. Response Rates 

The response rate were higher in the early 90’ s and dropped to 34 % in 1997 




51% 54% 50% 48% 49% 53% 47% 38% 34% 
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III. Nine years of data on the percentage of students using each payment source for 
their education at CSU. Table presented using percentages. The columns do not add to 
100% as students could select more than one option. 























Personal earnings 


62.6 


63.5 


61.7 


60.7 


55 


57.8 


54.3 


57 


55.3 


Parents supported 


48.6 


50.6 


51.7 


54.9 


56.4 


50.7 


48.8 


47.7 


44.7 


Spouse supported 


7.8 


8 


9.1 


6.6 


6.1 


5.4 


5.7 


6.6 


7.2 


Veterans benefits 


3.6 


4.3 


3.8 


3.4 


3.4 


3.8 


4.5 


5.7 


5 


Employer 

reimbursement 


9.9 


10.4 


10.9 


10.3 


10.1 


12.3 


12.1 


11.5 


11 


Private scholarship 


7.2 


7.1 


7.1 


6.7 


7.3 


7.5 


7.4 


7.9 


9.4 


University scholarship 


4.3 


3.7 


6.4 


5.6 


7.2 


5.5 


8 


8.5 


8.3 


Financial aid incl. loans 


29 


25.6 


22.8 


24.2 


25.4 


27.3 


32.3 


37 


39.2 


Vocational 
rehabilitation or other 
agency 




1 


0.9 


0.9 


1 


1.3 


1.3 


1.8 


0.9 


Personal loans 


“ 


— 


- 


— 


— 


— 


— 


— 


9.3 



rv. Nine years data on financial aid type for those students who reported having financial 
aid while attending CSU. Table presented using percentage. The columns do not add to 
100% as students could select more than one option. 



1333 


nrrjfy pi ^ 


M 




















mm 




Student loan 


29.5 


23.4 


19.9 


21 


22.9 


25.4 


30 7 


34.1 


39.3 


Pell grants 


15 


13.5 


14.3 


13.3 


15.6 


14.2 


16 


17.2 


18.3 


Work study 


7.9 


6 


5 


4.9 


5.5 


6.6 


5.5 


7.6 


7.5 


State financial aid 


7.6 


6.1 


6.2 


7.1 


7.7 


7.1 


7.3 


8 


8.6 


Other federal aid 


3.9 


4.3 


4.1 


4.9 


4.8 


3.8 


6.3 


6.9 


8.2 


Tuition waivers 


— 


— 


3.5 


3.4 


3.5 


5.1 


5.2 


6.3 


6.7 
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Introduction 

Identifying peer institutions and appropriate strategic indicators providing valid 
comparisons between institutions is an underlying theme related to benchmarking 
institutional measures of efficiency. As such, the relationship between reported financial 
ratios has come under closer scrutiny, especially with changes introduced by Statement of 
Financial Accounting Standards (SEAS) No. 117. In particular, the Educational 
Expenditure per FTE, as defined in U.S. News and World Report’s 1998 Undergraduate 
Ranking Criteria & Weights (U.S. News and World Report), is affected by the extent to 
which operation and maintenance of plant and depreciation are allocated to the 
components of total educational costs, i.e., instruction, academic support, student services 
and institutional support, subsequent to the implementation of SEAS No. 117. 

In institutions where a larger portion of operation and maintenance of plant and 
depreciation are allocated as indirect costs to auxiliary enterprises, allocated operation 
and maintenance of plant and depreciation would have less of an impact on overall 
educational costs. Because institutions vary in what is included under auxiliary 
enterprises; which, if any, auxiliary enterprises are outsourced; as well as whether any 
indirect costs are allocated to auxiliary enterprises that are not outsourced; comparisons 
of ratios involving educational costs are not necessarily valid. In order to have valid 
comparisons involving educational costs per FTE across institutions, additional 
information is needed about auxiliary enterprises, outsourcing and allocation of indirect 
costs. 



Background 

A review of the literature indicates that very little research has been conducted 
regarding the effects of outsourcing auxiliary enterprises. The most extensive study was 
conducted by Lee White (White, 1997). This study reports means for various financial 
ratios, by type of institution (public/private) and by Carnegie classification, for self- 
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operated and vendor-operated auxiliary activities. However, no statistical analysis was 
performed and the financial data are for the 1994-95 fiscal year, prior to the adoption of 
SFAS No. 117. 

A review of the literature could not identify any research pertaining to the effects on 
inter-institutional comparisons of allocating indirect costs to educational expenditures. 

Other groups are attempting to provide inter-institutional comparisons of direct 
educational costs by program. One such group is those institutions participating in a 
FIPSE-funded study of instructional costs and productivity being carried out by the 
University of Delaware. Ratios developed by the National Study of Instructional Costs 
and Productivity (NSICP) provide institutions with information that allows inter- 
institutional comparisons between instmctional costs grouped by CIP codes which may 
represent either programs and/or departments. Comparisons provided by the NSICP for 
small and medium-sized private independent institutions are not satisfactory due to 
program portfolio diversity among the institutions and also the small number of private 
institutions participating in the study. This study is limited to instructional costs and does 
not include the other components of educational costs (student services, academic support 
and institutional support). Also, indirect expenditures are not separately identified as part 
of the NSICP study. Reasons for non-participation in the study range from small 
institutional research offices to a lack of time and money to collect the data necessary for 
participation. 

The purpose of the research study described below is to document the extent to which 
strategic indicators involving educational costs are affected by the different ways in 
which indirect costs are allocated within institutions. The research study further proposes 
to determine whether a relationship exists between an institution’s decision to outsource 
auxiliary operations and its decision to allocate indirect costs to auxiliary enterprises. 

The lack of consistency across institutions in the allocation of indirect costs to auxiliary 
enterprises has rendered some inter-institutional financial ratio comparisons meaningless. 
It is this problem that the proposed research will attempt to clarify. 

Rationale 

The extent to which an institution outsources its auxiliary enterprises can affect the 
extent to which indirect expenses are allocated to auxiliary enterprises and in turn, affect 
the extent to which indirect costs are allocated to the functional classifications within 
educational and general expenditures. When selecting peer institutions to compare 
strategic indicators, it may be helpful to know the extent to which institutions outsource 
auxiliary enterprises. In addition, practices among private institutions differ in what is 
included in the category of auxiliary enterprises, as well as whether indirect expenses 
other than depreciation are taken against the revenues generated by auxiliary 
expenditures. 

Statement of Financial Accounting Standards (SFAS) No. 117, Financial Statements 
of Not-for-Profit Organizations, recommends that private institutions allocate 
expenditures relating to operation and maintenance of plant and depreciation among the 
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functional expenditure classifications and auxiliary enterprises. Assuming most 
institutions would comply with those guidelines, schools in our study were not simply 
classified as to whether they allocated indirect expenditures, but as to whether they 
allocated indirect expenditures other than depreciation. Subsequent to the adoption of 
SFAS No. 1 17, with the allocation of operation and maintenance of plant and 
depreciation among the other categories of expenditures, there may be an even wider gap 
in strategic indicators of institutions, depending upon the extent to which auxiliary 
enterprises are outsourced. 

The research study examined the relationship between strategic indicators involving 
educational costs and the outsourcing of auxiliary enterprises. Three research questions 
were investigated. 

1. Are institutions that outsource auxiliary operations less likely to allocate indirect costs other than 
depreciation to these operations? 

2. Is there a difference in mean Educational Expenditures per FTE between schools 
that outsource and those that do not outsource? 

3. Are there increased differences in Educational Expenditures per FTE following the adoption of SFAS 
No. 1 17, as reflected in the FY97 IPEDS Finance Survey data, between schools that do/do not 
outsource? 



As part of the research study information was collected about what operations were included in auxiliary 
enterprises, and which operations were outsourced. Additional information about the percentage of 
students in residence halls, the size of the endowment, and tuition was collected to help clarify factors 
contributing to Educational Expenditures per Fl'H. 

Sample 

The 340 four- year private colleges and universities that are members of EACUBO 
(Eastern Association of College and University Business Officers) were sorted by 
Carnegie classification and FTE enrollment based on IPEDS Enrollment Survey data. 

The 77 members classified as specialized institutions were excluded. Auxiliary revenue 
data from the FY96 IPEDS Finance Survey was downloaded from the Internet to 
determine which of the institutions had non-zero values for auxiliary revenues and 
expenditures. Only institutions with non-zero auxiliary revenues and expenditures were 
included in the study. After group membership (Carnegie class by size) was determined, 
a stratified random sample of 121 institutions was selected for participation. Business 
officers at each institution were contacted and invited to participate in the study. Of those 
contacted, 1 10 agreed to participate by completing a telephone survey. Examination of 
enrollment and auxiliary revenue and expenditure data revealed three outliers with 
enrollment less than 200 and unusual auxiliary data, leaving 107 institutions. The FY97 
IPEDS Finance Survey was requested from the institution since these surveys would not 
be publicly available until late 1998. Only eight institutions refused to release this 
information, resulting in complete data for 99 institutions. Follow-up telephone calls 
were made to clarify responses inconsistent with data reported on the IPEDS Finance 
Survey. 

The sample by Carnegie classification consisted of 29 Bachelor’s I, 29 Bachelor’s II, 
27 Master’s I, 6 Master’s II, 8 Doctoral and 8 Research institutions. Because of the small 
number of Doctoral, Research and Master’s II institutions, the Master’s I and II 



institutions were collapsed into one group, and Doctoral and Research became one group 
for some analyses. 



Survey Instrument 

Using the survey instrument, data were collected regarding which operations were 
included under auxiliaries, and which of these operations, if any, were outsourced. 
Questions about the allocation of indirect costs related to the types of indirect costs 
allocated (e.g., depreciation, maintenance, utilities, etc.), the allocation basis used (e.g., 
square footage) and to which operations indirect costs were allocated. 

Follow-up telephone calls were made where large differences between FY96 finance 
data and FY97 finance data were noted. A few of the institutions reported data 
incorrectly on the IPEDS Finance Survey, reporting in thousands of dollars instead of in 
whole dollars. Follow-up telephone conversations also were made to clarify seeming 
inconsistencies between data on the IPEDS Finance Survey and data obtained during the 
telephone interview. Several business officers responded that their school did not 
allocate depreciation to auxiliary enterprises, yet information on the IPEDS Finance 
Survey clearly showed that depreciation was allocated. Other institutions did not know 
which indirect costs were allocated because the auditors made the allocation. In a few 
cases, there were major shifts in data reported in the finance data between FY96 and 
FY97 and the information was determined to be reported accurately. 

Results 

In describing which operations institutions included under auxiliaries, the reader 
should note that an institution may have a particular operation on its campus and yet not 
include it under auxiliary operations. For example, if a day care center is operated only 
for the convenience of faculty staff and students, but is not used for educational 
experiences, then it probably is included as an auxiliary operation. On the other hand, a 
day care center used by the early childhood education department as a site for students’ 
practicum experiences would be classified as educational. In Table 1 below operations 
included as part of auxiliaries are listed along with the number of schools reporting. 
Operations included by at least one institution were facilities rentals, stores other than 
bookstores, golf courses, computing, camps, recycling, transportation, theatres, 
housekeeping, laundry, press/publications, and orientation/commencement fees. 



Table 1 



Operations Included in Auxiliaries 


Ooeration 


Number of Schools* 


Food Services 


104 


Residence Halls 


104 


Bookstore 


94 


Vending 


74 


Printing 


24 


Athletics 


22 


Parking 


21 


Health Services 


12 


Day Care 


1 1 


*N = no 




Outsourcing Operations 




Of the 110 institutions that responded to the telephone survey, 22 reported that no 
auxiliary operations were outsourced. There was no clear pattern based on Carnegie class 


or enrollment that would predict who was 


more likely to outsource operations. 


Examination of the percentage of resident students showed that most (18) of these 22 
institutions had primarily resident students ranging from 60% to 97%. A moderate 
correlation (r = .43, p < .0001) between percent of residents and not outsourcing was 
found. Among the remaining 85 institutions a variety of auxiliary operations were 
outsourced. These included food services, bookstore, vending, printing, parking, 
facilities rentals and cable television. The number of schools that outsourced each 


operation is shown in Table 2 below. 




Table 2 




Auxiliary Operations Outsourced 





Ooeration 


Number of Schools* 


Food services 


75 


Bookstore 


45 


Vending 


45 


Printing 


4 


Parking 


1 


Facilities Rentals 


1 


Cable Television 


1 



*N=110 



Of the 85 schools that outsource at least one operation, 39 outsource both the 
bookstore and food services, another 36 only outsource food services but not the 
bookstore, five outsource the bookstore and not food services and four only outsource 
vending and one only outsourced printing. Not surprisingly, 75% of institutions with 
food service in auxiliaries outsource food services, while only 48% outsource the 
bookstore. Operations not outsourced by any of the institutions in the sample were 
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residence halls, athletics, health services and day care centers. 

Allocation of Indirect Costs 

As shown in Table 3 below, there is a clear relationship between those schools that do 
not outsource food services and those that allocate indirect costs other than depreciation 
to food services. This result was not unexpected since an institution that provides its own 
food services, i.e., does not outsource, is more likely to accumulate all costs associated 
with food services, whether direct or indirect. What was an unexpected finding is that 
69% of those who outsource food services (and therefore do not have any direct 
expenditures for food services) still allocate indirect expenses to food services. Since 81 
of the 103 institutions were found to be allocating indirect expenses other than 
depreciation to food services, the impact of outsourcing food services was expected to be 
less important in affecting changes in the Educational Expenditures per FTE after the 
adoption of SEAS No. 1 17 as reflected in the FY97 IPEDS Finance Survey. 

Table 3 

Food Services Outsourced By Allocation of Indirect Expenses 

Food Services Outsourced? 



Allocate to food services? 


No 


Yes 


Total 


No 


3 


19 


22 


Yes 


25 


56 


81 


Total 


28 


75 


103 



X^ = 2.59,#=l,p=.09 



As shown in Table 4 below the results of comparing the outsourcing of the bookstore 
operation with those who do/do not allocate indirect costs are quite different than for food 
services. Although 64% allocate indirect costs independent of whether the bookstore is 
outsourced, exactly half of those who outsource bookstore allocate indirect costs. 

Again, since there is no strong relationship between outsourcing and not allocating 
indirect costs, the effect of SFAS No. 1 17 on Educational Expenditures per FTE would 
not be as great as originally expected. 

Table 4 



Bookstore Outsourced By Allocation of Indirect Expenses 





Bookstore Outsourced? 


Allocate to Bookstore? 


No 


Yes 


Total 


No 


12 


22 


34 


Yes 


38 


22 


60 


Total 


50 


44 


94 



X^ = 6.85,#=l,p<.01 
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Impact on Educational Expenditures 

The means for Educational Expenditures per FTE for FY96 and FY97 by Carnegie 
class are shown in Table 5 below. Recall that the effect of implementing SEAS No. 117 
would predict an increase in Educational Expenditures for FY97 since the dollars in 
Operation and Maintenance of Plant were allocated among other expenditure categories, 
including instruction and auxiliaries among others. Note the variation among the 
Carnegie classes in Educational Expenditures per FTE, especially the Research I and II 
institutions. The increase from FY96 to FY97 is expected, and is fairly consistent except 
for the group of 6 Master’s II institutions and the Doctoral and Research institutions. One 
might argue that any change in the ratio could be attributed to a change in FTE as well as 
a change in educational expenditures. This hypothesis was tested and for the sample in 
this study enrollment change in FTE was shown not to have an effect between 1996 and 
1997. Although the means for each Carnegie class increased from FY96 to FY97, three 
institutions actually showed decreases in Educational Expenditures per FTE. Each of the 
decreases was less than $1,000. 



Table 5 

Mean Educational Expenditures per FTE FY96 & FY97 



By Carnegie Class 

Mean Educational Expenditures per FTE 



Carnegie Class 


N 


Fiscal Year 1 996 


Fiscal Year 1 997 


Bachelors 1 


29 


$17,240 


$20,376 


Bachelors II 


29 


10,934 


12,584 


Masters 1 


27 


9,596 


11,565 


Masters II 


6 


8,776 


9,798 


Doctoral 1 & II 


8 


17,438 


22,474 


Research 1 & II 


8 


39,956 


52,293 




10 


$14,998 


$18,405 



The mean Educational Expenditure per FTE for all institutions that outsourced any 
operation for FY96 was $12,455 and it increased by $2,625 in FY97 to $15,080. For 
institutions that did not outsource any auxiliary operations the Educational Expenditure 
per FTE for FY96 was $14,541 which increased by $2,224 to $16,765 in FY97. The 
difference between groups is not significant; confirming that outsourcing is not the only 
factor to consider in accounting for change in Educational Expenditures per FTE 
following the implementation of SFAS No. 1 17. 

Based on earlier results showing that institutions were just as likely to allocate 
indirect expenses other than depreciation regardless of whether an operation was 
outsourced, a repeated measures AN OVA with two independent factors was run. One 
independent variable was allocation of indirect costs to food services and the second level 
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was outsourcing of food services. The results shown in Table 6 below show that 
differences in Educational Expenditures per FTE from FY96 to FY97 (Time) are 
significant (F = 28.2, df= 93,1; /? < .000) for the sample. Allocation of indirect costs 
other than depreciation is also significant (F= 9.57, df= 93,1;/? < .005), confirming the 
results on food services outsourcing and allocation. Note too that outsourcing of food 
service is not significant by itself or in its interaction across FY96 and FY97. 

Table 6 

Repeated Measures Analysis 

Educational Expenditures Per FTE FY96 and FY 97 



Within Ss 


df 


Mean Square 


F 


Time 


1 


68,823,819 


28.19"' 


Time x Allocate other than depreciation 


1 


16,520,168 


6.77* 


Time x Outsource food service 


1 


155,237 


.064 


Three-way interaction 


1 


4,432,645 


1.82 


Error 


93 


2,441 ,345 




Between Ss 


Allocate other than depreciation 


1 


632,975,417 


9.57** 


Outsource food services 


1 


100,335,545 


1.52 


Allocate x Outsource food services 


1 


171,525,927 


2.59 


Error 


93 


66,160,678 





p<.01 

**p < .005 

*"p< .0001 



A regression analysis was run in an attempt to account for variation in the change of 
Educational Expenditures per FTE from FY96 to FY97. Independent variables used in the 
research study such as outsourcing, allocation of indirect costs, Carnegie class and size 
were used. The results were not significant with less than 25% of the variance accounted 
for by this model 

The results indicated that other variables would be more closely related to educational 
expenditures than those selected. Such variables would include those relating to an 
institution’s ability to spend more on educational expenditures per FTE enrollment, such as 
full-time tuition price per student, net tuition per FTE enrollment, and the market value of 
the institution’s endowment per FTE enrollment. Another factor contributing to an 
institution’s increase in educational expenditures is the magnitude of an institution’s FY96 
expenditures for operation and maintenance of plant, which would be allocated among 
other expenditure categories for fiscal year 1997. The following correlation matrix 
supports these hypotheses. 
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Table 7 



Correlations 





Ed. 


Ed. 


Academic 


Tuition 


Net 


Net 


Mkt. Value 




Costs 


Costs 


Reputation 


Price 


Tuition 


Tuition 


Endowment 




per FTE 


per FTE 


Score 


97-98 


per FTE 


per FTE 


per FTE 




1996 


1997 


(USNWR) 




1996 


FY97 


6/30/96 


Ed.Costs/FTE FY97 


.977 














Academic Reputation 


.564 


.544 












Tuition Price 97-98 


.609 


.602 


.635 










Net Tuition/FTE FY96 


.558 


.577 


.541 


.804 








Net Tuition/FTE FY97 


.564 


.547 


.639 


.713 


.877 






Endowment/FTE 


.752 


.728 


.688 


.523 


.386 


.533 




Plant Exp/FTE FY96 


.716 


.727 


.432 


.446 


.342 


.374 


.686 



All Significant beyond .001 

A post hoc exploratory analysis using stepwise regression was conducted in an attempt 
to account for variation in Educational Expenditures per FTE for FY97. Independent 
variables used in the full regression model were selected from those variables shown to be 
highly correlated with education expenditures per FTE. Those selected were: market value 
of endowment per FTE, plant expenditures per FTE, 1997-98 full-time tuition price, and 
academic reputation score from US News & World Report 1998 College Rankings . The 
results were significant with 67% of the variance accounted for by the first three variables 
listed above. Endowment per FTE accounted for the majority of the variance, with 51%. 
Only academic reputation score did not add significantly to the explained variance. Further 
research will continue to investigate the relationships indicated in the above analysis. 

Conclusions 

Based upon the results of the research with the sample of 99 institutions, it appears that 
institutions that outsource auxiliary operations would not be any less likely to allocate 
indirect costs other than depreciation to these operations. There is no significant difference 
in mean educational expenditures per FTE between schools that outsource and those that do 
not outsource. Although there were significant increases in educational expenditures per 
FTE from FY96 to FY97 for the sample, allocation of indirect costs other than depreciation 
was found to be a significant factor, rather than outsourcing auxiliary operations. An 
institution’s policy on allocation of indirect costs, rather than outsourcing, should be 
considered in selecting peer institutions. 

A post hoc analysis revealed significant relationships between measures of institutional 
resources, such as endowment and revenues from tuition, and expenditures for operation 
and maintenance of plant. Academic reputation score was also found to be highly 
correlated with these measures. 
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MATH ACHIEVEMENT AND MATH COURSE-TAKING 

Denise Nadasen 

Coordinator, Institutional Research 
University System of Maryland 



INTRODUCTION 

Educational research has identified many factors that relate to student 
achievement - one of which includes math achievement. Math achievement is often 
associated with academic and professional success. Students with a strong grasp of 
mathematics are more likely to go to college and have an advantage in the job market 
(U.S. Department of Education, 1997). 

Despite our understanding of the importance of math achievement, U.S. students 
have not performed as well in math as students in other countries of similar political and 
economic conditions (U.S. Department of Education, 1997). The Third International 
Math and Science Study (TIMSS) by the U.S. Department of Education indicates that 
U.S. fourth grade students performed above the international average in math in 1993, but 
achievement of the middle and high school students were consistently below the 
international average. U.S. students performed particularly poorly in geometry compared 
with students from the 21 countries that participated in the study (U.S. Department of 
Education, 1998). 

A growing area of interest in math achievement research is study of students’ 
course-taking patterns. Some studies have shown that math coursework is related to 
student success. Research from the 1990 National Assessment of Educational Progress 
(NAEP) suggest that math coursework is related to high math achievement (Viadero, 
1993). The NAEP study analyzed the content of courses and found a link between test 
scores and specific topics discussed in math classes. Algebra and geometry were key 
topics that were related to math achievement. 

A more recent paper produced by the U.S. Department of Education, "Math 
Equals Opportunity", (U.S. Department of Education, 1997) finds that 8th grade algebra 
is a critical factor in predicting students likelihood of attending college. Nearly 83% of 
students who took algebra I went to college, while only 36% of those who did not take 
algebra I went to college (U.S. Department of Education, 1997). The study also revealed 
that 60% of students who took algebra I went on to take calculus in high school. 

A few studies have examined the relationship between taking more rigorous 
coursework and student achievement (ACT & CGCS, 1998; U.S. Department of 
Education, 1996). Data from the 1992 NAEP suggest that students who enrolled in 
higher-level math courses "had consistently higher average proficiencies" than students 
who were not enrolled in higher-level math courses (U.S. Department of Education, 

1996). 

The Council of the Great City Schools and the ACT Inc. examined student 
achievement of urban test takers. The data indicate that ACT test-takers who took more 
rigorous courses did better on ACT tests in all subject areas than those who did not. In 
particular, students who took trigonometry and calculus scored 30% higher on 
achievement tests than those who did not. Students who completed only the core math 
courses averaged 16.2 math achievement score, while students who completed 
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trigonometry and calculus averaged 22.7. Although this study is not generalizable to the 
larger population of students due to the selection process, the results suggest that all 
students can benefit by taking advanced math courses (ACT & CGCS, 1998). 

Research suggests that rigorous coursework can benefit students of all racial and 
gender groups equally, however a gap still exists between low and high socio-economic 
groups. Research shows that students from more affluent communities are more likely to 
be high achievers and that private school students tend to do better in math (Peng, 1994; 

& Gamoran, 1996a). Another study by Gamoran (1996b) suggests that students in inner 
city schools tend to lack the expectations, trust, and interpersonal obligations that most 
high achievers enjoy. 

What is the relationship between student course-taking patterns, student 
achievement, and socio-economic status? The ACT & CGCS study found that advanced 
coursework increased scores regardless of socio-economic status; however, students with 
a lower socio-economic background still lagged behind those of a higher socio-economic 
background (ACT & CGCS, 1998). The U.S. Department of Education (1997) found that 
students of a low-income background were much more likely to go to college if they had 
taken algebra I and geometry (U.S. Department of Education, 1997). However, the 
disparity between low and high socio-economic groups still exists. 

Table 1 

Percent of Students Attending College 
By Completion of Algebra I and Geometry 



SES 


Comnleted Ala 


I/Geom 


YES 


NO 


Low 


71 


27 


Middle 


84 


44 


High 


94 


60 



Source: U,S. Department of Education, 1997 

Even within the group that took the base-level math courses, a 33% difference exists 
between low SES and high SES students’ college-going rate. Is it possible that math 
courses have different effects on low and high socio-economic students? How do these 
differences play out in achievement scores? 

Course-taking is one of the strongest factors that schools have to influence student 
achievement; however, little research is available on the relationship between course- 
taking patterns and achievement. A recent study on course-selection procedures (Spade, 
Columba, & Vanfossen, 1997) suggests that high SES students have more advanced 
courses and a more elaborate course-selection procedure available to them. Spade, 
Columba, & Vanfossen examined case studies of six high schools in 3 socio-economic 
groups with both high and low performing standards. The study investigated the 
curriculum and curriculum selection in each school. A large disparity in the involvement 
of schoolteachers and counselors in the course-selection procedures exists between the 
affluent schools and the working-class and middle-class schools. Poorer students had 
fewer courses, less involvement with teachers and counselors, and less time to make 
selections than wealthier students (Spade, Columba, & Vanfossen, 1997). 
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Problem 

Students are typically required to complete two or three years of high school 
math to meet graduation requirements. Most students take only algebra I and geometry. 
Less than 20 percent of all students take high school trigonometry (U.S. Department of 
Education, 1997). 

This study examines the course-taking patterns of students in the U.S. and how 
patterns of course-taking relate to achievement scores for students of low and high socio- 
economic backgrounds. The study also examines the relationship between math 
coursework and math achievement. 

Course-taking analysis is usually complicated by the task of capturing, 
quantifying, and equating course information across school districts. Few national 
databases exist that include course data and math achievement for analysis. The U.S. 
Department of Education has collected this data in the National Educational Longitudinal 
Study of 1988 (NELS88). 

Using the NELS88 data, this study examines the relationship between math 
course-taking patterns and math achievement. First, this study will investigate students’ 
math course-taking patterns and their corresponding math achievement scores — for the 
total population and by socio-economic status. Do the course-taking patterns differ for 
each group? If so, are these differences related to achievement differences? 

Second, 1 introduce a model of analyzing coursework and math achievement. 

After adjusting for differences in students’ race, sex, and prior math achievement, 1 will 
determine how math courses contribute to math achievement. In particular, which math 
courses contribute the most to math achievement? Finally, do math courses contribute 
comparably to math achievement for both low and high socio-economic groups? 

METHOD 

Data Source 

The data for this study come from the U.S. Department of Education’s National 
Education Longitudinal Study of 1988, administered by the National Center for 
Educational Statistics (NCES). In 1988, NCES began collecting data on a representative 
sample of 25,000 8th graders in the U.S. Follow-up studies were completed every two 
years identifying students’ progress through and beyond high school. The sample used 
for this analysis consists of 16,489 students drawn from the base year (8th grade) and the 
second follow-up (12th grade) studies. In the base year, the first follow-up, and the 
second follow-up, the Educational Testing Service administered achievement tests in 
reading, science, and math. In addition to achievement test scores, information on family 
characteristics and course units were also collected. Many other variables were collected; 
all of which cannot be discussed here. Although some data were missing, pairwise 
deletion of cases and imputation of missing items by subgroup were used to complete the 
dataset. The survey had a response rate exceeding 90%. 

Variables 

The primary variables of interest are math course-units completed, math 
achievement scores, and socio-economic status. Secondary variables of interest include 
race, sex, and prior math background. Because of the longitudinal nature of the NELS88 
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data, this analysis is designed to control for demographic variables and prior math 
experience so as to isolate portions of math achievement that may be attributable to math 
coursework. 

The course-taking patterns are identified using course variables: algebra I, 
geometry, algebra II, trigonometry, and calculus, from the second follow-up study. Each 
course variable is a composite of the number of Carnegie units in math completed by 
each student. One course unit is the equivalent of one full year of coursework. Course 
units vary from 0 to 4, with the average around 1.0. Students who completed at least 
some of the course were considered course-completers. Calculus and precalculus are 
combined into one variable because students usually take one or the other. 

The 12th grade math achievement scores are used as the criterion variable to 
measure the effects of math coursework. Standardized scores are typically used 
nationally by college admission boards as a measure for comparing student performance. 
Approximately 3,000 students from the NELS 88 study did not complete the 12th grade 
standardized math achievement test in 1992. The missing data spanned all race, gender, 
and SES groups, indicating no obvious bias in the missing scores. Imputation of the 
means, by each of the subgroups, was used to complete the data set. 

Three demographic variables are used in this analysis: race, sex, and socio- 
economic status. All three are composite variables that combine questions that have been 
drawn from several surveys in the NELS88 study. Socio-economic status is used as a 
comparison variable to look at how the relationship between math course-taking and 
math achievement may differ for each group. SES is grouped into 4 quartiles. Only the 
first and fourth (highest and lowest) quartiles are used in this analysis. The low SES 
group includes students in the lowest quartile of the combined SES category and the high 
SES group includes students in the highest quartile of the combined SES category. 

(Thus, students in the middle were excluded from the comparison.) To adjust out 
differences that may be due to race and sex, the regression analysis uses these two 
variables as covariates. 

The 8th grade math achievement scores are also used as a covariate to reconcile 
differences in different math backgrounds. Correlation analysis indicates that 8th grade 
math achievement is highly correlated with 12th grade math achievement. By controlling 
for prior math experience, math achievement differences can be more distinctly 
attributable to course-taking patterns. 

Analysis 

Correlation analysis explores the relationships between the 12th grade math 
achievement and the 8th grade math achievement, the 5 course variables, race, sex, and 
socio-economic status. 

Course-taking patterns of various groups are identified by the frequency of 
students who completed units in each of the five high school math courses. Students who 
have some course units are considered completers. Students who have zero units are 
considered non-completers. For each group within each course, an average math 
achievement score is reported. The same data are reported for the low SES group and the 
high SES group. 

Forward regression analysis is used to compare seven incremental math course- 
taking models. The dependent variable is 12'^ grade math achievement. Race, sex, and 
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prior math achievement are the covariates. The five math courses are the independent 
variables. First, race and sex are entered into the equation, followed by the 8th grade 
math achievement. The five math courses are entered on next five steps in the order in 
which they would be taken: algebra I, geometry, algebra II, trigonometry, and 
calculus/precalculus. 

From the regression results model summary, the change in the amount of 
explained variance (R square) is calculated. The change in explained variance 
attributable to each model provides a meaningful measure of comparison for each course. 
In addition, a method typically known as "ordered regression" is used to determine the 
significance of the amount of change in explained variance contributed by each variable. 
The incremental increase in the sum of squares in the analysis of variance is measured. 
Using the residual from the final model, a new mean square and F statistic is calculated 
for each model. 

RESULTS 

The correlation matrix in Table 2 demonstrates the strength of the relationship 
among the five course variables, prior math achievement, and the 12th grade math 
achievement scores. All correlations are significant at the .01 level. Prior math 
achievement and 12th grade math achievement have the highest correlation. Algebra 1 
has the lowest correlations with prior math achievement, trigonometry, and calculus. 

(See table 2.) 



TABLE 2 

CORRELATION MATRIX 





PRIOR MATH 


ALG I 


GEOM 


ALG II 


TRIG 


CCALC 


MA ACH 


PRIOR MATH 


1.00 


0.26 


0.42 


0.40 


0.35 


0.48 


0.76 


ALG 1 


0.26 


1.00 


0.68 


0.50 


0.26 


0.26 


0.27 


GEOM 


0.42 


0.68 


1.00 


0.64 


0.37 


0.38 


0.46 


ALG II 


0.40 


0.50 


0.64 


1.00 


0.30 


0.33 


0.43 


TRIG 


0.35 


0.26 


0.37 


0.30 


1.00 


0.37 


0.37 


CCALC 


0.48 


0.26 


0.38 


0.33 


0.37 


1.00 


0.50 


MA ACH 


0.76 


0.27 


0.46 


0.43 


0.37 


0.50 


1.00 



NOTES: All correlations are significant at the 0.01 level (2-tailed). PRIOR MATH = 8 "' grad math achievement; 
MA ACH = 12th grade math achievement score; CCALC = combined pre-calculus and calculus. 



o 

ERIC 



97 






The 12th grade math achievement test has an overall average of 50.4 and a 
standard deviation of 8.9. The averages by course participation for the total sample, the 
low SES group, and the high SES group are provided in Table 3. 

TABLE 3 

AVERAGE MATH ACHIEVEMENT SCORE 
BY COURSE ENROLLMENT 



all students = 16>396 





no 


Average 


yes 


average 


Difference 


algebra 1 


5,120 


47 


1 1 ,276 


52 


5 


Geometry 


8,097 


46 


8,299 


54 


8 


algebra II 


10,245 


47 


6,151 


55 


8 


Trig 


14,037 


49 


2,359 


58 


9 


ccalc 


13,939 


49 


2,457 


61 


12 



low ses students = 3,867 





no 


Average* 


yes 


average 


Difference 


algebra 1 


1,808 


42 


2,060 


46 


4 


geometry 


2,774 


43 


1,093 


49 


6 


algebra II 


3,082 


43 


786 


51 


8 


trig 


3,687 


44 


181 


53 


9 


ccalc 


3,705 


44 


163 


56 


12 



high ses students = 4,271 





no 


Average 


yes 


average 


Difference 


algebra 1 


913 


56 


3,358 


57 


1 


geometry 


1,258 


54 


3,014 


58 


4 


algebra II 


1,941 


55 


2,330 


58 


3 


trig 


3,159 


55 


1,113 


60 


5 


ccalc 


2,976 


54 


1,295 


62 


8 



Note: ccaJc combines pre-catcutus and calculus: approximately 400 
out of the total 1,460 took both courses. 



The seven-step regression produced significant F-statistics at all steps. However, in order 
to evaluate the significance of the incremental change in the explain variance due to each 
variable, ordered regression analyses were conducted. Table 4 presents the results of the 
ordered regression. The sum of squares (SS) and degrees of freedom (DI^ represent the 
increase from the previous model. A new mean square (MS) and F statistic (adjusted for 
design effect) are calculated using the residual from the final model. Significance levels 
were determined by the .05 alpha level. In all models prior math experience provided the 
largest F-statistic, as expected. All coursework provided significant contributions to 
math achievement in the Analysis of Variance table for all students. In the models for the 
low SES students, all courses except trigonometry showed significance. The high SES 
students showed significance for all course gains except algebra I and algebra II. 



100 

o 

ERIC 



98 



TABLE 4 

ORDERED REGRESSION RESULTS 

ANOVA TABLE FOR ALL STUDENTS 



Model 


SS 


DF 


MS 


F* 


Sig 


Race/Sex 


16661.55 


2.00 


8330.78 


283.39 




Prior Math 


702735.76 


1.00 


702735.76 


23905.34 


* 


Algl 


6617.16 


1.00 


6617.16 


225.10 


* 


Geom 


23937.74 


1.00 


23937.74 


814.30 


* 


Algll 


6853.43 


1.00 


6853.43 


233.14 


« 


Trig 


5854.43 


1.00 


5854.43 


199.15 


« 


Ccalc 


13740.03 


1.00 


13740.03 


467.40 


* 


Residual 


463410.65 


15764.09 


29.40 







ANOVA TABLE FOR LOW SES STUDENTS 



Model 


SS 


DF 


MS 


F* 


Sig 


Race/Sex 


1996.10 


2.00 


998.05 


11.15 




Prior Math 


69363.61 


1.00 


69363.61 


774.92 


* 


Algl 


3399.25 


1.00 


3399.25 


37.98 




Geom 


3525.71 


1.00 


3525.71 


39.39 




Algll 


3119.53 


1.00 


3119.53 


34.85 




Trig 


228.46 


1.00 


228.46 


2.55 




Ccalc 


1329.63 


1.00 


1329.63 


14.85 


* 


Residual 


91152.50 


3655.83 


24.93 







ANOVA TABLE FOR HIGH SES STUDENTS 



Model 


SS 


DF 


MS 


F* 


Sig 


Race/Sex 


494.58 


2.00 


247.29 


2.61 




Prior Math 


128683.08 


1.00 


128683.08 


1358.66 


* 


Algl 


102.80 


1.00 


102.80 


1.09 




Geom 


2073.75 


1.00 


2073.75 


21.90 


* 


Algll 


286.66 


1.00 


286.66 


3.03 




Trig 


1191.68 


1.00 


1191.68 


12.58 




Ccalc 


6968.14 


1.00 


6968.14 


73.57 


* 


Residual 


108751.72 


4122.12 


26.38 







Note: ccalc combines pre-calculus and calculus: approximately 400 out of 1,460 students took both courses. 
F’ is adjusted for the design effect for sample to match population proportions. Sig =. 05; fcrit=3. 84(1 dF). 



Table 5 displays the amount of explained variance in the final regression models 
for all students, low SES students, and high SES students. The change in R-square is the 
amount of variance in 12‘*' grade math achievement scores explained by the variable(s) 
entered in that step. It measures the effects of the independent variables on the dependent 
variable. In all three tables, prior math achievement accounts for the greatest amount of 
explained variance. For all students, geometry contributes the greatest amount of 
explained variance of the five course variables. In the low SES models, algebra I, 
geometry, and algebra II contribute the most to 12"’ grade achievement scores. For the 
high SES models, calculus explains the greatest amount of variance in grade 
achievement scores. Although minuscule, changes in R square are generally found to be 
significant. (Most likely due to a large sample size.) 
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TABLE 5 

REGRESSION MODEL SUMMARY 
Change in R Square Due to Course Variable 



All students 



Model 


R 


R Square 


Change in R-sq 


Sig. 


Race/Sex 


0.116 


0.013 


0.013 




Prior Math 


0.762 


0.580 


0.567 




Algl 


0.765 


0.586 


0.005 




Geom 


0.778 


0.605 


0.019 




Algll 


0.781 


0.610 


0.006 




Trig 


0.784 


0.615 


0.005 




Ccalc 


0.791 


0.626 


0.011 





Low SES students 



Model 


R 


R Square 


Change in R-sq 


Sig. 


Race/Sex 


0.107 


0.010 


0.010 




Prior Math 


0.640 


0.410 


0.400 




Algl 


0.660 


0.429 


0.019 




Geom 


0.670 


0.450 


0.021 




Algll 


0.680 


0.470 


0.020 




Trig 


0.690 


0.470 


0.000 




Ccalc 


0.690 


0.480 


0.010 


* 



High SES students 



Model 


R 


R Square 


Change In R-sq 


Sig. 


Race/Sex 


0.045 


0.002 


0.002 




Prior Math 


0.721 


0.520 


0.518 


* 


Algl 


0.721 


0.520 


0.000 




Geom 


0.727 


0.528 


0.008 


* 


Algll 


0.728 


0.530 


0.002 




Trig 


0.731 


0.534 


0.004 


* 


Ccalc 


0.750 


0.562 


0.028 


* 



Note: ccalc combines pre-calcuius and calculus: approximately 400 



out of the total 1,460 took both courses. * = sig. at .05 alpha. 



DISCUSSION 

The combined set of math courses explains 4.2% of variance in the 12th grade 
achievement scores. This finding is lower than other research findings (ACT &CGCS, 
1998). The differences may be, in part, due to imputation of missing data, which reduces 
the variance that exists in the actual data set. Of the five courses, geometry contributed 
the most to 12th grade achievement scores, and algebra I had the smallest effect on the 
scores. Of the covariates, 8th grade achievement scores had the highest correlation with 
12th grade achievement. 

The correlations among the variables are significant. Often if the predictor 
variables are highly intercorrelated, the threat of multicollinearity exists. 

Multicollinearity affects the sampling variance of the standardized coefficients in the 
prediction equation. However, data sets with large sample sizes are generally much less. 
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sensitive to the effects of multicollinearity. 

Consistent with the findings fronn the regression analyses, the highest correlation 
exists between prior math achievement and 12th grade math achievement: .76 (See Table 
2). Geometry is highly correlated with algebra I and algebra II. The lowest correlations 
(although still significant) exist with algebra I: prior math achievement, trigonometry, 
calculus, and 12th grade math achievement. 

The results of the course taking analysis suggest the following (Table 3): 

- Approximately 22% of students who take algebra I will go on to take 

calculus in high school. 

- For high SES students, 39% of students who take algebra I will go on 

to take calculus. 

- For low SES students, approximately 8 % of students who take algebra 

I will go on to take calculus in high school. 

- While 90% of high SES students move on to geometry after algebra I, 

only 53% of low SES students enroll in geometry after algebra I. 

The differences in course-taking patterns between low SES students and high SES 
students are striking. The data are particularly troublesome considering the impact on 
math achievement scores. A 12-point gain in achievement scores can be realized when 
comparing low SES students who completed calculus and low SES students who took 
only algebra I. Low SES students who took algebra I, on the average, earned 4 points 
more than students who did not take algebra I. High SES students measured a 1 -point 
difference between those who did take algebra I and those who did not take algebra I. 

The overall gain for high SES students who completed calculus when compared with 
those who completed only algebra I is 8 points. This data imply that the gains for low 
SES students can be much greater than those for high SES students . 

Some courses did affect scores of one group differently than the other group. 

Low SES students benefited more from taking geometry and algebra II than 
trigonometry, whereas, high SES students gained more from geometry and trigonometry 
than from algebra II. This may be explained by differences in the quality of the courses 
taught at the different schools. Regardless of SES status, however, taking advanced math 
coursework early will help all students improve their math achievement scores, but low 
SES students have a greater opportunity for improvement than high SES students. 

As previous research has indicated, math achievement gains are realized when 
rigorous math courses are taken: the more difficult the coursework, the greater the gain 
(ACT & CGCS, 1998). The results of this study show that the overall changes in R- 
square are somewhat low, but significant (See Table 5). For all groups of students, 
calculus (combined precalculus and calculus) improves on math achievement scores on 
the average by 3 points. Geometry also provides a large gain for all students. Geometry 
explains 2.2% of variance in 12^^ grade achievement scores, while algebra I, algebra II 
and calculus add 1 % each. Trigonometry fared modestly (but significantly) among the 
coursework variables. 

Consistent with other findings in this study, the amount of explained variance 
from coursework differed for low SES and high SES students. Low SES students realize 
the greatest gains from the algebra I, geometry, and algebra II sequence, while high SES 
students realize the greatest gains from calculus. 
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CONCLUSION 

The purpose of this research is to examine the relationship between math course- 
taking patterns and 12th grade math achievement scores for all students in the sample and 
for low and high SES groups separately. The most surprising results show that low SES 
students have the potential for greater gain than high SES students when taking more 
rigorous math coursework. Although the gap continues to exist, it does narrow as 
students take more match coursework. 

The analysis identifies courses that students are likely to take and how those 
course-taking patterns affect math achievement. Taking advanced math courses 
contributes significantly to math achievement scores - even after adjusting for differences 
in race, sex, and prior math achievement. The fact that prior math achievement is highly 
correlated with 12'*’ grade math achievement and explains a significant amount of 
variance implies that math achievement begins before students enter high school. Gains 
in math achievement are realized for all students, regardless of socio-economic status, but 
specifically for students who take geometry and calculus. 

This research has implications for policy makers and curriculum developers, 
particularly for poorer school districts. Curriculum developers can offer more courses 
that are shown to improve achievement, providing students with more opportunities to 
benefit from rigorous coursework. Policy-makers may require all students, particularly 
students in low SES area, to complete a rigorous math course sequence as part of the 
general high school requirements. Clearly opportunities exist for poorer students to 
improve math performance and benefit from greater success later in life by changing 
course-taking patterns. 
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USING CLUSTER ANALYSIS FOR TRANSCRIPT ANALYSIS OF 
COURSE-TAKING PATTERNS 



Wayne S. Obetz 
Research Associate 
Office of Institutional Research 
Community College of Philadelphia 



Until recently. Community College of Philadelphia (CCP) offered to its students 
an Associate’s in General Studies degree. Over thirty percent of its associate degree 
recipients over a recent ten-year period graduated through this curriculum (Annual 
Statistical Compendium, 1995). Requirements for graduation included the completion of 
three English courses, one additional humanities elective, two social science electives, 
two Science/Math electives and twelve free electives. Because of the variety of courses 
available to students in this curriculum, it was difficult to understand completely the 
nature of the course choices made by General Studies graduates. 

While the reasons a student chose to enroll in a particular mix of courses might 
have been known to the student, and perhaps to his or her advisor, and those reasons 
might have been firmly based in the most sound pedagogical reasoning, the college did 
not systematically attempt to understand or leam from the patterns of enrollment 
displayed by its General Studies graduates. This was unfortunate because the college 
missed out on an opportunity to leam from its students what they wanted from the college 
that was not otherwise offered by its more structured curricula. 

There was a specific need to know more about the following issues; 

1) Is there a common set of courses taken by all General Studies graduates? Does 
a core curriculum exist for this program? Are these graduates systematically 
avoiding some courses and taking others? 

2) Are General Studies graduates completing second-level courses, or are they 
completing only introductory courses? Is there a balance between breadth of 
studies and depth of study? 

3) Outside of the common core of courses, are there patterns in the course-taking 
of the General Studies graduates? Are there distinguishable subsets of 
graduates, with unique patterns of study? Do these patterns make sense, i.e., 
are they clearly directed towards transfer or a career of some sort? 

Typically, questions of this sort have been answered through the use of transcript 
analysis. Transcripts are completely objective and unobtmsive measures. Although they 
do not record information about out-of-classroom experiences, attitudes, and 
commitments, they do give us an accurate accounting of the courses, grades, and degrees 
a student completes (Adelman, 1992, 1995). The process of transcript analysis 
traditionally has been a manual one and is costly, time-consuming, and subject to biases 
of sampling and of those people sorting through the assembled transcripts. 

Adelman (1990, 1992, 1995) made use of two national databases of transcript 
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information to detail what was studied and where by graduates of the high school classes 
of 1972 and 1982. These studies were undertaken to better understand the different 
curricular experiences of students at various types of institutions, and how those 
experiences changed over time. While Adelman’s work provides a detailed descriptive 
look at the individual courses and degrees completed by two cohorts of students, it does 
not provide us with information about the combinations of courses taken by any group of 
students. Adelman’s approach can be used to provide answers to the first two research 
questions, but does not address the issue of patterns raised in the third research question. 

A different way to approach this question is through the use of cluster analysis. 
This technique makes it possible to reduce a very large body of data to a relatively 
compact description (Anderberg, 1973). In the college setting, it has been used by 
student counseling professionals to more efficiently target programming (Jones & 
Pinkney, 1991) and by researchers to link specific patterns of coursework to learning 
achieved (Ratcliff, 1992). 

In their book. Principles of Numerical Taxonomy, Sokal and Sneath (cited in 
Aldenderfer & Blashfield) argued that cluster analysis could be used to place relatively 
similar organisms into groups and these groups could be analyzed to determine if they 
represented biological species. In much the same way, the course-taking patterns of 
students can be clustered and then analyzed to determine if they represent some sort of 
informal curricular structure. 

Different clustering methods can generate different solutions to the same data set 
(Aldenderfer & Blashfield, 1984). Anderberg (1973) cited this as a strength of the 
technique. A single classification may give a distorted view of a multifaceted data set. 
There may in fact be many meaningful groupings and a variety of cluster analysis 
techniques will be needed to reveal them. Everitt (1993) wrote: 

A variety of alternative classifications for the same set of objects or 
individuals will always exist. Some classifications will be more useful 
than others. . ..The important point is that any classification is a division of 
the objects or individuals into groups based on a set of mles - it is neither 
tme nor false (unlike say a theory) and should be judged largely on the 
usefulness of the results, (p. 3-4) 

In many applications it might be reasonable to apply a number of 
clustering methods. If all produce similar solutions, the investigator might 
perhaps have more confidence that the results are worthy of further 
investigation. Widely different solutions might be taken as evidence 
against any clear-cut cluster structure. Comparison of different 
classifications is clearly of some importance here. (p. 141-142) 
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Procedure 



Data Acquisition and Preparation 



I requested the following data from the CCP Computer Center for all students 
who graduated with an A.G.S. degree between Fall 1984 and Spring 1992: 



Student I.T>. num6er 

‘Entering year and term 

Entering curricuCum 

JinaCyear and term 

EinaC curricuCum 

OdumSer of semesters attended 

Eitst year and term zuitfi credit 



‘Department and course numBer of eacB. 
course enroBCed in (ma 7 (imum of SO 
courses) 

grade for each course enro[[ed in 



Records for 1957 A.G.S. graduates were returned by the Computer Center. All records 
were examined for the presence of failed courses and duplicate courses. All failed 
courses and all but the final occurrence of duplicate courses were eliminated from the 
analysis because these courses do not carry credit towards graduation. 

Each graduate’s course-taking history was analyzed to determine if they 
successfully completed 80% or more of the required core courses for any of the 60 
defined curricula of the college. This was done to remove them from further analysis 
since the stated purpose of the study was to determine the underlying curricular structures 
of graduates’ course-taking patterns. If students “shadow” another of the college’s 
curricula, their course-taking patterns are already understood. It is economical and 
relatively riskless to remove from the data set those with obvious patterns as they are 
found and concentrate on the remaining, more confused residue (Anderberg, 1973). This 
process eliminated 92 graduates from the analysis. 

Similarity Coefficient and Clustering Techniques 

The variables used for comparisons of all remaining graduates were completed 
courses. Jaccard’s coefficient was calculated for all pairwise combinations of graduates. 
This measure was used because it gives no weight to joint absences. When the simple 
matching coefficient is used, some cases appear very similar primarily because they both 
lack the same features rather than because the features they do have are shared. In 
contrast, Jaccard’s coefficient is concerned only with features that have positive co- 
occurrences (Aldenderfer & Blashfield, 1984). It might be unreasonable to consider two 
individuals as very similar simply because they both lack a large number of qualities 
(Everitt, 1993). Students may enroll in any of the nearly 700 courses offered by the 
college, and graduates need to complete less than 30 courses in order to qualify for 
graduation. It makes little sense in this case to give equal weight to the 650 or so courses 
that a pair of graduates jointly did not take as it does to the 50 or so courses they did take. 

Graduates were cluster analyzed using a variety of clustering methods in SPSS for 
Windows Release 7.0. Hands and Everitt (1981) suggested that Ward's method was the 
best at recovering true cluster structure on a data set of the type employed in this study 
when the data contained approximately equally sized clusters, but centroid clustering was 
superior when the clusters were of different sizes. Since there was no reason to suspect 
the clusters under investigation were more likely to be of equal sizes than unequal sizes. 
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nor vice-versa, both of these hierarchical agglomerative methods were employed. 

Milligan’s findings (cited in Aldenderfer & Blashfield, 1984) suggested that 
^means clustering, using an initial starting partition derived from average linkage 
clustering, should also be examined. Therefore, the K-Means Cluster Analysis routine in 
SPSS was employed with initial cluster centers derived from the average linkage between 
groups method. This clustering routine was employed for all solutions ranging from the 
2-cluster solution to the 50-cluster solution. 

Assessing Clustering Solutions 

Although a number of heuristic procedures and formal tests have been developed 
for determining the “ optimal” number of clusters, each of these procedures is marked by 
reliance on subjective decisions by the researcher (Aldenderfer & Blashfield, 1984; 
Everitt, 1993), or require the analyst to defer Judgement to the computer (Anderberg, 
1973). Anderberg argued that: 

It is comforting to formulate the problem as a search for optimality 
because when the solution is found it can be eagerly adopted without 
further ado.. ..An alternative to optimality is to use heuristics and intuition 
to find solutions “by inspection.”. ..[This] approach enhances the efficacy 
of subjective Judgement rather than surrendering to a completely specified 
procedure before the first candidate solutions are seen. Rather than 
remove human judgement altogether, the idea is to focus Judgement on 
plausible alternatives, (p. 24). 

For purposes of this analysis, the optimal solution for each method was one that 
provided a number of large clusters to investigate while simultaneously accounting for as 
many graduates in the sample as possible. A minimum cluster size of 40 members was 
chosen because it allows for an average of five graduates a year for each of the eight 
years of the study. It can be argued that a program of study that on average provides five 
graduates a year to the college is both vital and viable. I decided that the large clusters 
for each method should account for at least 75 percent of all graduates. I felt that 
allowing more that one-quarter of the sample to fall into small, unexamined clusters 
would lessen the utility of a solution. The solution selected for each method can be seen 
as an attempt to strike a balance between the desire to maximize the number of large 
clusters and the need to minimize the loss of students accounted for by those clusters. 

For each of the three relevant clustering methods, cluster memberships for each 
subject were saved for all solutions ranging from the 2-cluster solution to the 50-cluster 
solution. Tables were then developed for each clustering method showing cluster 
frequencies for each of the 49 different solutions. For each solution, the number of 
clusters with at least 40 members was calculated, as well as the total number of graduates 
and percentage of the overall sample that these large clusters accounted for. These tables 
were used to determine the most useful solution for each clustering method. 

Comparing and Profiling Classifications 

Once the clusters for each solution were assigned, they were compared to the 
clusters assigned by the other methods to determine the overlap on clusters. A three-way 
crosstabulation of the cluster memberships was calculated for this purpose. In the present 
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case, those graduates who were identified as members of the same group by all three 
methods were considered first for further analysis. This was done by examining the cells 
of the crosstabulation and choosing those groups large enough to be considered useful. 
Once again, 40 members was chosen as the minimum group size. 

This method did not account for all large clusters from the three clustering 
solutions initially chosen. Therefore, the 3 two-way crosstabulations of the cluster 
solutions were examined for those clusters not represented in the three-way analysis. 

This yielded additional groups for analysis, but once again did not account for all large 
clusters. These remaining clusters were examined on their own merits. This means of 
choosing groups for analysis may seem roundabout, but Anderberg (1973) argued: 

The clusters are not interesting in themselves at all; the point of interest is 
in inferences about the structure of the data.. ..This explanatory structure is 

the object of the search Once a satisfactory structure is known and 

defended on its own merits, any cluster analysis that contributed to its 
discovery is only of historical significance, (p. 19) 

Each group of graduates was profiled as to their most frequently completed 
courses. This was done both for General Studies graduates in general and for all groups 
identified through the above procedures. Groups were also profiled on number of transfer 
credits, years of enrollment, and final curriculum. These profiles were used to draw 
inferences about the intent of those graduates in each group. 

Results 

Choosing Cluster Solutions 

Ward’s method provided the easiest choice between cluster solutions given the 
selection criteria. Because the tendency of this method is to create approximately equally 
sized clusters, it was possible to select the 1 1-cluster solution and still account for 100% 
of the graduates. This provided a fair number of clusters to investigate and had the added 
benefit of accounting for the entire population. 

For the centroid method, the 42-cluster solution provided the maximum number 
of large clusters while still accounting for at least 75% of the graduates. Although it 
would have been possible to account for more of the graduates by selecting a solution 
with fewer large clusters, I decided to maximize the number of clusters rather than 
minimize the loss of graduates accounted for. This was done because all graduates had 
already been assigned to one of the 1 1 large clusters of Ward’s method. This solution 
provided seven large clusters that accounted for 79.9% of the graduates. 

Three candidate solutions were examined for the k-means method: (a) the 31- 
cluster solution, with 9 large clusters accounting for 95.3% of the population; (b) the 33- 
cluster solution, with 10 large clusters accounting for 95.1% of the population; and (c) the 
50-cluster solution, with 14 large clusters accounting for 93.1% of the population. When 
the large clusters from the 31-cluster solution were used in the three-way crosstabulation, 
more graduates were collected together in the large groups formed in the cells of the 
crosstabulation, and fewer of its clusters failed to enter into one of the large groups than 
did the clusters formed by the other two k-means solutions. For these reasons, the 31- 
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cluster solution was used for all further analyses. 

Comparing Classifications 

The three-way crosstabulation of classifications yielded 12 groups of 40 or more 
graduates. These 12 groups accounted for 1029 (55.2%) of the 1865 graduates analyzed. 
Of the 7 large clusters identified by the centroid method, 6 were included in at least one 
of the 12 groups identified; 10 of the 1 1 large clusters identified by Ward’s method were 
included in one of the groups; and 7 of the 9 large clusters from the fc-means analysis 
were included in one of the 12 groups. 

The 3 two-way crosstabulations yielded four more groups. Unlike the groups 
formed by the three-way crosstabulation, membership in one of these four groups did not 
preclude membership in one of the remaining three groups. Twenty-seven graduates 
appeared in three of the groups, and 7 graduates appeared in two groups. Once the 
overlap across groups was factored out, these groups accounted for an additional 209 
graduates. The two large clusters from the /:-means analysis that were not included in 
any of the groups of the three-way crosstabulation were used here, however there 
remained one large cluster each from both of the hierarchical analyses which had not yet 
entered into any of the 16 groups identified. 
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These two clusters were considered groups in their own right. The 40 graduates 
who were identified by the remaining Ward’s method cluster had not been identified as 
members of any of the other earlier defined groups, but 34 of the 43 graduates in the 
remaining centroid method cluster had membership in the groups identified by the two- 
way crosstabulations. Over 70% of the graduates either were set aside from further 
analysis or entered at least one of the groups identified (see Table 1). 

Profiling Groups 

The 18 groups identified are presented here in decreasing order of the 
apparentness of their course-taking patterns. Many of the groups appear to have similar 
types of interests and are therefore presented together. 

Groups 7, 8, 9, and 10 all had clear health career interests. Members of these 
groups tended to complete several biology courses, and many completed chemistry 
courses. The nursing curriculum was the one in which they were most likely to have 
completed the greatest percentage of course requirements. Other Allied Health programs 
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were also represented in these course taking patterns, but not to the same extent. 

Members of Group 12 were likely to take chemistry, biology, and physics 
courses, but unlike those in Groups 7 through 10, they were likely to have completed the 
greatest percentage of course requirements of the chemical technology curriculum. This 
combination of courses and curricular interest possibly indicate an interest in further 
studies in pharmacy. 

The most popular courses for those graduates in Group 1 1 have an international 
emphasis; World Literature I & II, World Civilization 1 & II, World Religions, The 
Individual in the World, Elementary Spanish, Elementary French, and Cultural 
Anthropology. Each of these courses was completed by 90 % or more of the members of 
this group. Of the 70 members of this group, 63 completed more of the requirements of 
the pre-education curriculum than they did of any other curricular offering of the college. 

Group 18 similarly appeared to have a core set of courses that were taken by the 
majority of group members. All members of this group took the six most popular 
courses, and the two next most popular courses were taken by 98 % of the group. 
Amongst these eight courses there was an advanced literature course, a course in 
American culture, and a philosophy course. This pattern corresponds to the courses 
generally taken by those students who participate in the honors program of the college, a 
program directed towards transfer to a four-year college. 

Groups 6, 13, 14, 15, 16, and 17 had apparent business interests, and these 
interests took several forms. In Groups 15 and 17, 12 of the 50 most popular courses 
were office administration offerings. Introductory data processing and introductory 
accounting courses were likewise popular with these two groups. Those in Group 6 were 
more likely to be enrolled in marketing, management, accounting, and economics courses 
than those in other groups. Group 16 members tended to complete more computer- 
related program requirements than other program requirements. Groups 13 and 14 took 
courses from throughout the various business offerings, but did not concentrate their 
efforts to quite the same degree as did those in the other groups. 

The pre-education program was overwhelmingly the program in which the 
greatest percentage of course requirements were completed for Groups 1 and 5. The 
members of these two groups on average completed fewer courses with prerequisites than 
did members of any of the other groups. 

Members of Groups 2, 3, and 4 had varied curricular interests as evidenced by the 
variety of curricula in which they completed the greatest percentage of course 
requirements. They were as likely to have completed these requirements in a social 
services program as in a business or education program. There is no immediately 
apparent core set of courses for these groups. The few courses that were popular with 
these groups were introductory courses spread over various content areas and were the 
same courses popular with all General Studies graduates, regardless of group. 



Discussion 



There was no typical pattern of course-taking for the overall population of 
General Studies graduates. That being said, there also were a number of graduates whose 
studies had a clear pattern to them. In this study, four identifiable course-taking patterns 
were evident. 

“Shadowing” of Restricted Enrollment Curricula 

Many students are unable to gain admission to their first choice curriculum. 
Because of the high costs associated with the programs in Allied Health, the college 
accepts only a fraction of the students who apply to these programs. In an attempt to 
avoid oversaturation of the market, and in order to make the practicum placement process 
a more manageable one, the college limits enrollments in other programs, such as Early 
Childhood Education. Although some students get shut out from these select curricula, 
they do not abandon study in these areas. 

For example, those General Studies graduates who closely followed a pattern of 
study usually associated with Allied Health curricula very often had a final curriculum 
code indicating they either were waiting to get into an Allied Health curriculum, or had 
already been in one. Their course selections reflected their desire to graduate from one of 
these programs. These students either (a) failed to qualify for admission to the program 
of their choice; (b) were at some point admitted to their desired program, but could not 
complete the requisite course of study; or (c) changed their mind about gaining admission 
to a particular program, but only after shadowing its curricular requirements for a period 
of time. Although they were unable to complete all the requirements for graduation from 
one of the Allied Health curricula, they were able to complete most of the coursework 
and still maintain eligibility for graduation from the General Studies curriculum. 

Like the Allied Health curricula, enrollments in Early Childhood Education are 
restricted. And just like those who patterned their course selections after Allied Health 
curricula requirements, failure to gain admission to Early Childhood Education (or failure 
in one of its required courses once admission had been gained) did not prevent these 
students from pursuing the course of study they desired. This refusal to abandon a 
chosen course of study is one of the ways students use the institution in what Adelman 
(1992) referred to as “making their own history.” Despite the roadblocks constructed by 
the college and paths leading away from their intended course of study, these students 
would not be deterred. 

Shadowing of Business Curricula 

Finding an explanation for the shadowing of the business curricula of the college 
presents a more difficult problem. These programs are open access, but there are reasons 
a student might prefer to graduate through General Studies rather than a business 
curriculum. If the student intends to transfer to another college, specific courses required 
by a more structured curriculum might not transfer. In order to minimize loss of credits 
then, the savvy student will take all the courses that transfer easily, and fill in with 
electives. The General Studies curriculum was very accommodating of this practice. 

The College recognized this practice, and put in place business transfer curricula tailored 
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to those colleges to which its students were most likely to transfer. 

Another reason a student might have elected the General Studies route was to 
avoid a difficult required course. This is the type of practice that can be best observed 
and remediated by an advisor working one-on-one with an individual and his or her 
transcript. The advisor must be able to sense which curriculum most closely matches an 
advisee’s course-taking to that point and encourage full participation in that curriculum. 

Novel Course-Taking Patterns 

There were a number of graduates whose course -taking patterns did not reflect the 
requirements of any of the defined programs of the college. Instead, these students took a 
collection of courses that had personal relevance. One group surveyed the college’s 
offerings in world civilizations and cultures. A second group appears to have been 
concentrating their studies in courses offered in the honors program of the college. A 
third group took a number of biology, chemistry, and physics courses while completing 
the greatest percentage of course requirements in the chemical technology curriculum. 
According to a member of the chemistry faculty at Community College of Philadelphia, 
many of these students continue their studies in pharmacy. 

The course-taking patterns of these students should be telling the college 
something. While some of their peers were taking courses that did not yield an 
immediately apparent pattern, the three groups described above took courses zuranged 
around an educational core. These cores did not shadow another curriculum of the 
college and therefore should be seen as an opportunity for the college to offer to its 
students a program of studies which is both educationally sound and in obvious demand. 

Unfocused “Patterns” 

The college must redouble its efforts to understand the educational intent of those 
whose course-taking did not yield a recognizable pattern. Absent the ability to find the 
structure underlying the course selections of these students, the relationship between the 
courses they were enrolled in and stated outcomes for their program would be difficult, if 
not impossible, to demonstrate. The degree the college confers upon these students must 
be as meaningful to the college as any of the other degrees it awards. The tendering of a 
degree should not be automatic upon the completion of a required minimum number of 
credits but rather confirmation that an acceptable program of study has been completed. 

Conclusions 

Cluster analysis techniques are useful in the search for course -taking patterns, but 
the method has its limitations. As Aldenderfer and Blashfield (1984) and Anderberg 
(1973) cautioned, clustering methods always place objects into groups, and these groups 
may be “real” or merely an artifact of the measures and methods employed for a given 
analysis. There is a human art to making meaning of the groups the techniques uncover. 

Although this study concerned graduates of the General Studies curriculum at 
Community College of Philadelphia, in no way is the technique limited to that 
population. It can be used to reveal patterns of course-taking within more structured 
curricula, and it can be used at high schools, four-year colleges, and graduate schools as 
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well. To the extent that elective coursework is allowed, students will use the freedom 
accorded to them to bend the curriculum in many unanticipated ways. Faculty and 
administrators should seize the opportunity to learn from the patterns evident in the 
course-taking of their students, and to ensure the educational quality of the collection of 
courses in which their students ultimately enroll. 
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The original problem 

In the mid-1980’s, the academic audit process at Community College of Philadelphia 
(CCP) was beginning to settle into a set of somewhat predictable patterns. Those programs 
identified for audit would come to the Office of Institutional Research with requests for 
information needed for the completion of the audit. In almost all cases, the requests were for 
the same core set of information. Every year would see about five new programs added to 
the list of those actively conducting audits. Most audits took more than a year to complete, 
so there was a need to provide updated information to many programs each year. 
Additionally, faculty and staff from other areas of the college would show up in the office 
with requests that began to fit their own predictable patterns. It became obvious that 
providing customized information for each of these requests was an unmanageable way of 
doing business and a search for a reasonable solution began. 

The data elements that were at the core of the majority of requests became the logical 
starting point for structuring the database we planned to put together. Almost all programs 
under audit were interested in some key demographics of their students — age, race, gender, 
and geographic area of the city the student resided in, among others. There was usuily 
interest in a similar set of “input” variables — placement test scores, entering 
developmental/remedial status, high school attended, etc. There was, of course, interest in 
the “process” variables — e.g., curriculum enrolled in, retention patterns, participation in 
educational support programs, campus attended, full-time/part-time status. None of the 
evaluations would have been complete without the standard “outcome” variables — GPA, 
credit hours earned, graduation status, academic probation status. 

The solution 

What we had then was a list of variables, some which were static (e.g., race and 
gender) and others that would either change (e.g., cumulative GPA) or take on new values 
(e.g., semester GPA) with the passage of time. It was clear that there would be two parts to 
the database: the “static” demographic portion and a “dynamic” semester-by-semester 
portion. Over time, the importance of collecting certain information, which to that point had 
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not been collected, became apparent, and those data elements were added to the database. As 
each semester passed, the semester portion of the database required updating. New 
information was appended to the end of each existing record, and new records were added to 
the database. Soon we had a large database, with lots of information, but we came to realize 
that it was difficult to manage in its originally conceived form. 



Data elements on the database specification^ 

Demographic record 



Social Security number 

Race 

Gender 

Birth year 

ZIP code 

Certificate curriculum, year & term 
awarded 

Associate’s curriculum, year & term 
awarded 

English 101 grade, year & term completed 
English 102 grade, year & term completed 
English 112 grade, year & term completed 
Reading test score 
Writing test score 



Math test scores (old form) 

Math test scores {new form) 

Test waiver code 
Previous college indicator 
Credits transferred in 
Credits by exam 
High School code 
High School graduation year 
First year and term enrolled at the college 
First year and term attempted credit 
courses at the college 



Semester record 



Social Security number 
Semester (year and term) 

Curriculum code 

Session (Day, evening, weekend, etc.) 
Full-time/Part-time status 
Educational Support Service enrolled in 
Credits withdrew from during semester 
Regional Center indicator 
Financial aid indicator 
Semester hours attempted (credit hours 
only) 

Semester hours earned (credit hours only) 
Semester GPA 

Semester registered hours (includes 



developmental courses) 

Cumulative hours attempted (credit hours 
only) 

Cumulative hours earned (credit hours 
only) 

Cumulative GPA 

Cumulative registered hours (includes 
developmental courses) 

Probationary status 
Clock hours (includes non-credit and 
developmental courses) 

Residency code 
Business and Industry code 
Community Services room 



* Data elements appearing in italics were added to the file at various points and were not present in the first 
version of the database specification. 
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Community Services curriculum 
Original structure of the file 

The original specification for the file called for a single record to be entered on 
the database for each student (see Figure 1). This longitudinal record consisted of a 
demographic block at the front with all the information about a student that would not 
change over time, or could take on only one value of interest (e.g., English 101 grade 
from the most recent attempt at the course). Attached to the back end of this block were 
semester “buckets”. These buckets were filled with the same information for each 
semester of a student’s attendance. If a student did not attend for one of the semesters for 
which a bucket had been allocated, an empty bucket was placed in the record. This was 
necessary because the position of the bucket in the record was the indicator of which 
semester’s information was being examined. As each new semester was added to the 
database, a new bucket was simply appended to each student’s record. By the time ten 
years of data were available on the database, there were 40 buckets (4 semesters per year 
X 10 years). Most of these buckets were empty for most of the students. A second 
problem crept up when a new piece of information had to be added to the demographic 
block or to each semester’s bucket — it forced all information in following semesters to 
shift to the right, and all the data list statements for our SPSS code had to be modified to 
account for this change, as did all our QUIKJOB code. A new approach was indicated. 

Current structure of the file 

The rectangular, case-ordered structure of the original file proved to be too 
inefficient and too difficult to update. Fortunately, data file structures are not limited to 
that simple approach. SPSS supports complex, non-rectangular files. The solution we 
decided upon was to build one of those file types, a nested data file. Files of this type 
contain various record types with a hierarchical relationship among the record types. A 
record of one type may have its information distributed across records of other types. 

This file type allows us to build a set of records for each student, with the demographic 
information being one of those records, and each semester of attendance having a record 
of its own (see Figure 2). The demographic information can be distributed across each 
semester’s worth of information, and analysis can proceed from there. This database 
structure eliminates the waste of empty buckets, and allows for the efficient addition of 
new information to each record — it can simply be appended to the end of each record 
without affecting the information in each following semester for that student. 

Emerging issues 

The broader-than-anticipated use of the data puts issues on the table that were not 
part of the original design criteria for the database. In addition to the more generalized 
use the Office of Institutional Research has found for the file, other administrative units 
of the College access the data contained in it rather than the primary sources they 
previously used. This has presented the following challenges: 

a) The importance of fully operationalizing all definitions of the elements used in the 
data file. When access to the file is no longer restricted to those who came 
together to design it, it is imperative that a common understanding of what the 
individual elements represent be agreed upon, and that understanding must be 
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based upon objective standards acceptable to all parties. 

b) Defining who has the responsibility for the necessary validity and reliability 
checks. If the data in the warehouse are retrieved from other administrative areas 
of the college, who should check for inconsistencies in the data? Who should 
change the data, and should those changes ripple back through the system to the 
files they were created from? 

c) The creation of our data warehouse has fundamentally changed the nature of the 
relationship of the Office to the Computer Center. The implications of this 
change have effected the operations of the Office, both in terms of the types of 
information we now request from the Computer Center and in the ways data is 
analyzed. 



Figure 1 



Original file layout 
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Current file layout 
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THEY’LL SURF BUT THEY WON’T SWIM: 

STUDENT RELUCTANCE TO APPLY TO COLLEGE ONLINE 
AND IMPLICATIONS FOR WEB-BASED SURVEY RESEARCH 

Sarah Parrott 
Jennifer McKnight 
Tufts University 

Access to computers and the Internet has dramatically increased among college- 
bound students during the past few years. With this in mind, many college admissions 
professionals have begun to place college applications on the Internet (and in the case of 
the MIT Sloan School of Management, require students to apply on-line). Even with such 
technological advances that should make the application process easier and more 
convenient for all parties involved, student response to non-mandatory Web-based 
college applications has been lackluster at best. Why? 

The present study seeks to answer that question and shed light on many of the 
issues and concerns surrounding Internet-based inquiry in an institutional setting. 
Specifically, this study examines (1) how college freshmen use their computers and the 
Internet, and (2) the reasons why students are reluctant to use the WWW to apply to 
college. Because the Internet is a fairly new information medium and growing at such 
and extraordinary pace, little timely research exists that documents student usage 
patterns. Thus, this study uses information about how the current cohort of incoming 
freshmen is using computers and the Internet as a framework to examine their reluctance 
to apply to college online. Implications based on the findings are discussed in terms of 
both the college application process and Internet-based survey research. 
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Background 

A recent national study by the Art & Science Group showed that more than three out of 
four college-bound high school students (78 %) used campus Web sites while applying to 
college last fall, up from 58 % in 1997 and a mere 4% in 1996 (Academe Today, 1998). 
Most “surfed” the Internet to gather information about prospective colleges and 
universities. Fully realizing that the current generation of potential applicants is the most 
Interaet-savvy to date, admissions offices and software companies have eagerly 
attempted to take advantage of this boom in Internet use among the college applicant 
population. This is evidenced by the proliferation of college admissions Web pages 
designed to aid the college search process and equipped to handle online applications (see 
Academe Today, October, 1998). 

While online applications aim to ease the application process by reducing the workload 
for both students and admissions offices, both technological and perceptual obstacles 
have resulted the lackluster acceptance of online applications. The Art & Science Group 
study found that print catalogs are still the most important sources of information for 
college-bound students, and the same is true for traditional pen-and-paper applications. 
Their survey of 500 college-bound high school seniors showed that only 2 1 % preferred 
on-line college applications, down from 34% in 1997. The present study uses a large 
multi-institutional sample of college freshmen to take a closer look at Internet usage as 
well as the reasons why students are reluctant to apply to college online. 

Data and Sample 

Data used in this study were drawn from three surveys administered to separate 
groups of students who expressed interest in a top-ranked private liberal arts university in 
Spring, 1998. The first group surveyed consisted of students who inquired about 
admission to the university, but who did not subsequently send in an application 



(N=l,l 17). The second group was made up students who applied, were accepted, and 
chose not to attend the university (N=980). The third group consisted of applicants who 
were accepted and who chose to attend the university in fall, 1998 (N=893). Thus, the 
2,990 responses represent entering freshmen who were accepted at a variety of colleges 
and universities across the country. All respondents were asked specific questions about 
their computer and Internet use, and their concerns about applying to college 
electronically. 

There were no statistically significant differences between the three groups of 
respondents used for this study. White respondents constituted 62.2% of the overall 
sample, followed by Asian Americans (13.7%), Hispanics (7.3%), African Americans 
(5.7%) and Native Americans (0.2%). The rest of the sample (10.8%) marked “other” as 
their race or ethnicity. Women and men were fairly evenly divided with respect to the 
college population with 41.2% of the respondents male, and 58.8% female (national 
college populations are approximately 45% male and 55% female on average). 

Slightly more than half of the students in the sample came from public high schools 
(57.5%), while 35.5% attended private high schools, and 7% marked their high school 
type as “other”. Consistent with the high-achieving students who are attracted to top- 
ranked institutions, almost all (97.1%) aspired to a graduate degree. Many came from 
financially secure backgrounds as well — fewer than one in five students (18.3%) 
expressed concern about their ability to pay for a college education. 
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Results 



Student Internet Access 

Almost all the students surveyed (96.4%) had access to the Internet. This is not 
surprising, given that the factors most predictive of computer ownership and on-line 
access are family income and education. People with a college education are ten times as 
likely to own a computer compared to those without any high school (McConnaughey & 
Lader, 1997). Since parental college education is also a major determinant of college 
attendance for their children (Astin, 1993), it follows that the students who plan to attend 
college and express interest in high-ranked institutions most likely come from families 
who own computers and have on-line access. 

This was shown to be the case in this study. More than three out of four 
respondents had Internet access at home (85.2%), which is four times higher than the 
national average of 18.6%, and more than twice that of households where parents have a 
bachelor’s degree or above (38.4%)' ((McConnaughey & Lader, 1997). A large 
percentage of students (86.6%) had Internet access at school. Just over half (54.6%) had 
access at their library, and 32.6% had Internet access at their parent’s place of work. 

Interesting differences begin to emerge when access to the Internet is examined 
by race. Not surprisingly. White and Asian American students have significantly more 
access to the Internet from home than do African American and Hispanic students (see 
Figure 1). This is consistent with the findings from a recent report by the National 



' These percentages vary due to rural or urban status, income, state, age of householders, and household 
type. 
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Telecommunications and Information Administration (1997) which showed that even at 
incomes higher than $75,000, whites are significantly more likely to own computers than 
are African Americans and Hispanics. Even more disturbing is the finding that at the 
same income level, the rates of on-line Internet access nationally are nearly three times as 
high for whites (21.2%) as for African Americans (7.7%), or Hispanics (8.7%) 
(McConnaughey & Lader, 1997). 

Figure 1. 

Internet Access at Home by Race 



Internet Access at Home 




White American Asian Amencan Hispanic American African American 



Computer and Internet Usage Patterns^ 

For what specific tasks are college-bound students using their computers, and 
what are they doing online? Figure 2 shows that computers are being used most for 
word processing (98.2%) and Internet “surfing” (96.1%), followed by email (89.9%), 
video games (58.7%), art or graphics (50.7%), Internet discussion groups (22.1%), and 

^ These questions were only asked in two of three of the groups surveyed. N=l,873. 

1 - 2.7 
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shopping online (10.8%). Even though almost two of three students (62.4%) reported 
using the Internet to download college materials, only 16.3% reporting using the Internet 
to submit college applications. 

Figure 2. 

Computer Usage 




Interestingly, differences begin to emerge— particularly in applications that are 
Internet-based— when women and men are viewed separately (see Table 1). Men are 
significantly more likely than women to report frequent use of the Internet (71.3% 
compared to 57.8% of women — a 13.5% difference). They also report more frequent use 
of- Internet discussion groups (6.1% for men versus 2.8% for women), downloading 
college information (14.3% for men, 13.9% for women), and submitting college materials 
(3.7% compared with 2.3% among women). Interestingly, in a surprising reversal of 
gender stereotyping, freshman men reported shopping online twice as much as did 
women (1.7% of men compared to only 0.6% of women reported shopping online 
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“frequently”, and 14.8% of men and 7.6% of women said they shopped online 
“frequently” or “occasionally”). Men are also more likely to use computers for video 
games (35.2% compared to only 6.9% of women). The results from this sample of 
college freshmen with respect to video game usage is consistent with national findings 
from the 1 997 CIRP Freshman Survey which showed that men are significantly more 
likely to use computers for video games than are women (Sax, Astin, Korn, & Mahoney, 
1997). Women and men did not differ substantially in use of word processing or email. In 
fact, the use of email was the only Internet-based application where more women 
reported frequent use compared to men. 

Table 1. Computer and Internet Usage* 



Comouter aoDlication 


Men 


Women 


% Difference 


Video aames 


32.5% 


6.9% 


25.6% 


Internet surfina 


71.3% 


57.8% 


13.5% 


Art or araohics 


1 3.4% 


8.4% 


5.0% 


Internet discussion arouos 


6.1% 


2.8% 


3.3% 


Submittina colleae aoDlications 


3.7% 


2.3% 


1.4% 


ShoDoina online 


1 .7% 


0.6% 


1.1% 


Downloadina colleae materials 


1 4.3% 


13.9% 


0.4% 


Word orocessina 


93.9% 


94.0% 


-0.1% 


Email 


69:6% 


71.4% 


-1 .8% 



* Percent reporting “frequent” use. 



These results clearly indicate that among the current generation of college 
freshmen, men are more “wired” in terms of Internet usage than are women. The reasons 
for this gender gap in Internet usage is not entirely clear, however it may be that young 
men are encouraged to use computers and the Internet more than are young women. 
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More research needs to be conducted to explore the reasons why such a gender gap in 
access to the online information infrastructure exists. 

Electronic Submissions 

Even though well over half of the college-bound students in the sample reported 
downloading college materials from the Internet, it was shown earlier that fewer than one 
in five students submit college applications electronically. Among the students surveyed, 
the primary concerns about submitting college applications electronically include; 

• Fear that data submitted might be lost in transmission 

• Perception that the process was “too impersonal” 

• Dislike using credit card on the WWW 

Additionally, a large number of respondents also mentioned the user-friendliness of 
the WWW application as being problematic. For example, students mentioned they were 
not able to write in the margins on an electronic application like they were able to do on a 
traditional pen-and paper application. Also, they could not include additioneil information 
beyond the scope of the application. 

Respondents also expressed concerns about the security of their electronic 
applications (i.e., “data can be accessed by anyone.”), and some thought that admissions 
officers might perceive them as being “lazy” for applying electronically. They also 
expressed concern that such applications would not be given the same consideration as 
paper applications. 
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Crosstabulations were used to determine significant differences due to gender or 



race. Overall, women respondents expressed more concern about applying electronically 
than did men. Compared with men, they were significantly more likely to be concerned 
that (1) their application data might be lost in transmission, (2) that the WWW 
application process is “too impersonal”, and (3) that their personal computer could not 
handle the task. 

Table 2. Reasons for Not Applying Electronically by Gender 





Men 


Women 


% 

Difference 


Parents/friends pcrfer paper application 


15.1% 


11.1% 


4.0% 


Was not aware that submitting electronically was option 


11.2% 


8.3% 


2.9% 


Thought it might be a disadvantage 


12.6% 


10.4% 


2..2°k 


No comouter access 


4.5% 


4.2% 


0.3% 


Encountered technical nroblems 


5.0% 


5.5% 


-0.5% 


Too imoersonal 


35.8% 


37.7% 


-1.9% 


Do not know how application will loolc 


25.7% 


28.6% 


-2.9% 


Did not have computer skills necessarv 


5.5% 


9.7% 


-4.2% 


Personal comouter cannot handle the task 


6.4% 


11.1% 


-4.7% 


Do not like using credit card on WWW 


29.5% 


36.3% 


-6.8% 


Paoer seemed more reliable 


57.6% 


66.9% 


-9.3% 



The reasons given for not applying electronically indicate that women are not as 
comfortable with using the Internet to apply to college as are men. Not only do women 
report that their computer cannot handle the task of applying electronically than men, 
they express greater skepticism about the Internet as a whole. This is not surprising given 
the previous findings which show that men enter college with more experience in all 
areas of Internet use — with the exception of email - than do women. 

Other slight— although non-significant— differences were found with respect to 
race. Asian American students had more concerns about the electronic application 
process than other ethnic groups, including White students. Asian Americans were more 
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concerned than were other ethnic groups about the fact that the electronic application 
process would not allow them to see how their application would look before they sent it 
in, and expressed more hesitancy about using a credit card via the WWW. Additionally, 
almost half were concerned that the data might be lost in transmission (compared with 
roughly a third of White respondents). 

Implications 

The most striking finding from this study is that, even among this group of high- 
achieving students, women still lag behind men in terms of online use and willingness to 
take advantage of Internet-based applications. Obviously, men are being socialized more 
to accept computers and explore all aspects of the Internet. They "surf "the Internet more 
frequently, engage in online discussion groups more often, shop more online, download 
college materials to a greater degree, and also apply to college online more than women. 

This has important implications for both college admissions professionals and 
institutional researchers who plan to conduct web-based surveys. In terms of the college 
admissions process, it is probable that men will utilize this option more than will women. 
Additionally, based on the findings of this study, white students are more likely to use the 
medium of the Internet for college applications than are non-white students. This might 
be due to the fact that, even among higher-income households, white families are more 
apt to have a computer and online access in their home. 

The current trend toward mandatory online college applications must take these 
differences into account. Even though most students (at least in this somewhat privileged 
sample) have access to the Internet, we have to realize that there are a complex set of 



online behaviors that differ as a function of gender and/or race. Therefore, the prevailing 
assumption that online access automatically implies online usage is not entirely valid. 
Such differences may even be greater among students who do not aspire to top-ranked 
institutions. 

On the other hand, in terms of the applying to college electronically, many 
respondents in this sample mentioned user-interface issues as being problematic. This 
means that they were uncomfortable with the specific WWW college application itself, 
not necessarily with the concept of applying to college via the WWW. It is plausible that 
as the format for electronic applications become more user-friendly, students will become 
more amenable to using the Internet to apply to college. 

Suggestions for Web-based Survey Research 

The findings from this study also provide institutional researchers who plan to 
administer surveys on the WWW with useful information on how to optimize the 
effectiveness of online surveys. First, the gender differences found in this study point to 
the fact that men may well be more likely to respond to an online survey than women. 
This study showed that men spend more time online than do women, and have a greater 
range of experience with the different Internet-based applications. 

Based on the finding that women use email slightly more than men, using email to 
contact women about a web-based survey may be an effective way to encourage them to 
participate in the survey. Additionally, if a link to the survey site can be inserted into the 
email message itself, and easily accessed by the student, response rates may increase. 

The key point here is ease of accessibility to the survey site from an email platform. 
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Student concerns about online security and user-friendliness is directly applicable 
to Internet-based survey research and design. Based on the finding that women and 
minority students express more skepticism about the Internet, care must be taken to craft 
a web survey in such as way as to alleviate their fears about confidentiality and 
reliability. Respondents must be assured to the following upfront (before the survey 
instrument is presented): 

• The length of the survey, how many questions will be asked, and the time investment 
required to complete it 

• The confidentiality of the responses 

• Options to change responses, easily navigate through the survey, and control the final 
submission of the survey 

• Address and phone number of the office administering the survey 

• Any prizes or incentives being offered^ 

Additionally, the online survey instmments should include: 

• Plenty of room to for complete responses and comments after each item 

• Specific contact information displayed prominently for respondents to report 
problems or questions, or to request a paper survey 

• Automatic receipt of response after the survey is submitted, including a nice “Thank 
you!” 



^ A recent study about how people use the WWW suggested that offering incentives, prizes, or “freebies” 
resulted in a much better than usual response rate for Web-based surveys (Kiernan, 1998). 
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• Instructions on where to send additional information or obtain the results of the 
survey 

Because using the Internet for college applications and surveys is a relatively new 
phenomenon and little research has been conducted that has investigated computer and 
Internet use among students, this study offers a starting point for both researchers 
interested in student computer interactions and practitioners who are interested in using 
the Internet to conduct surveys in college or university settings. 



For further information about this study, please contact: 



Sarah Parrott or Jennifer McKnight 
Tufts University Office of Institutional Research 
28 Sawyer Ave. Medford, MA 02155 
(617) 627-3274 
sparrott@infonet.tufts.edu 
imcknight@infonet.tufts.edu 
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ON-LINE NEWS VS TRADITIONAL MEDIA: STUDENT PREFERENCE 
REGARDING THE ACQUISITION OF CURRENT EVENTS 

Dawn Geronimo Terkla and Jennifer McKnight 
Executive Director and Research Analyst, Office of Institutional Research 

Tufts University 



It is estimated that approximately 90 percent of college and university students in 
North America have ready Internet access, compared to less than one-tenth of the general 
population (Chidley, 1996).' Given that colleges and universities are now admitting 
students of the “NET generation”, it is imperative that institutions understand how their 
learners acquire information. There is a dearth in the literature that explores this 
phenomenon. In fact, it has long been espoused that university students are subject to 
potentially different patterns of information acquisition than typical adult samples and as 
a result little is known about how college students acquire information about significant 
news events (Slater, 1983). A few studies, completed in the early eighties, examined 
student preferences between television and newspapers (Barrows, Clark & Klein, 1980; 
Henke, 1998; and Perloff, et al, 1982). To date, there appear to be no published findings 
that have added electronic news services to the mix. 

In the summer of 1997, a contract was signed with Prentice Hall to obtain 
university-wide access to Simon & Schuster’s College NewsLink, a web-based 
newsclipping service with stories classified by academic discipline.^ As part of the 

' In a recent National Telecommunications and Information Administration report. Falling Through the 
NET II: New Data on the Digital Divide, the 1997 nation-wide data indicated that in American households 
18.6% had on-line access, 26.3% had modems, and 36.6% had personal computers (McConnaughy, Lader, 
Chin & Everette, 1998). 

^ The website address is http://www.ssnewslink.com 



license agreement, it was stipulated that the university would evaluate the product in 
terms of ease of access, reliability, desirability and usefulness. Simultaneously, there was 
interest on campus in determining how students prefer to receive their news information. 
In an era of limited resources, the question was being raised whether the university 
should use funds to provide multiple subscriptions to the New York Times or a daily 
electronic news service. This paper will describe how the institutional research office 
went about developing a strategy to evaluate College NewsLink, assess students’ 
preferences regarding the acquisition of current events, and the difficulties that were 
encountered along the way. 

METHODOLOGY 

The approach taken to evaluate this product was multi-pronged. First, a short 
survey was administered to all faculty members to ascertain level of familiarity with web- 
based products. Second, surveys were administered to students in the various classes in 
which NewsLink was incorporated into the curriculum. Third, faculty members who 
used NewsLink in their courses were interviewed; and fourth, specific questions were 
designed and incorporated into the 1998 Graduating Senior Survey. 

In October 1997, shortly after NewsLink became available for use on campus, a 
short survey designed to describe the product and gauge potential interest in 
incorporating the product into courses was distributed to all Tufts faculty. The survey 
asked five questions about familiarity and use of NewsLink, willingness to attend a 
training session, and use of the New York Times for teaching purposes. As with many 
faculty surveys, the response rate was very low at just 6 percent (N=76). 
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Students in courses where NewsLink was intentionally incorporated into the 
curriculum (NewsLink-enhanced courses) were queried to ascertain levels of usage and 
preferences for various web-based electronic information sources. One hundred forty- 
five students in five classes were surveyed. The courses were exclusively in the social 
sciences: Introduction to Psychology, Cognitive Aspects of Intergroup Relations, Survey 
of Social Theory, Judicial Politics, and Political Psychology. The survey instrument 
focused on student usage of electronic news services, students’ perceptions of the quality 
of two distinct web-based products, the magnitude of student usage of additional 
electronic tools, the types of resources utilized for research papers and projects, student 
newspaper reading behavior, and student preferences for obtaining current events. 

At the close of the spring 1998 semester, the five faculty members who 
incorporated web-based products into their courses were interviewed. The interview 
protocol was quite extensive. Faculty were asked to provide information regarding their 
satisfaction with the specific web-based tools, a description of how the tool was 
integrated into their curriculum, their perceptions of student use, and an assessment of 
how such tools could be made available to other faculty members in their department. 

The 1998 Graduating Senior Survey included a limited number of questions 
regarding the acquisition of current events. Specifically, students were asked four 
questions: 1) whether they had used the on-line electronic news services, 2) to identify 
the newspapers they read on a regular basis, 3) to identify the information sources that 
they used during the academic year to leam about current events, and 4) their preferred 
source of current events information. Members of the Class of 1998 were surveyed prior 
to May graduation. The overall response rate was quite high (approximately 97%). 
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FINDINGS 



Use of Electronic Resources 

Tufts undergraduates are definitely part of the NET generation. Almost 100 
percent of the students in the NewsLink-enhanced courses reported using email regularly, 
84 percent use the World Wide Web (WWW) for personal use, 73 percent use WWW for 
research, 49 percent use on-line library catalogues, and approximately 8 percent use 
listservs & newsgroups (Usenet). 



Percent who Use These Electronic Tools on a Regular Basis 




Email WWW for WWW for Onttno library Uaarwt Other 

personal use research catalogues 



Of those students who use on-line resources for research papers and projects, 
most (68%) use these resources in tandem with books and periodicals. In addition. 

References Sources Used for Research 



Mostly online 
research sources 
10% 




books/periodicals 



approximately 13 percent of the students in surveyed classes use the internet for research 
purposes as their main reference source. It is interesting to note that 43 percent of the 
respondents indicated that they had received training or asked for assistance in using 
Tufts electronic research tools. 

Examination of this population, albeit small, suggests that there are some 
significant differences between men and women in their approach to the use of electronic 
resources. More undergraduate men (92%) than women (78%) reported using the WWW 
for personal use. Undergraduate women reported using multiple-sources (both on-line 
resources and traditional resources, such as books and periodicals) for research purposes 
more frequently than their male counterparts (71% vs. 50%). Women were also more 
likely to indicate that they had received training for Tufts electronic research tools (48% 
vs. 37%). 

There appear to be very few differences between the various classes and their use 
of electronic resources. Our initial h)q)Othesis was that we would see more activity 
among the lower classmen than the upper classmen. Regardless of class year, all students 
seem to be using email with about the same frequency. A slightly higher proportion of 
first-year and sophomores reported using the WWW for personal use and research. 
However the differences were not significant. Juniors and seniors reported using the on- 
line library catalogues with greater frequency than first-years and sophomores. Thus, it is 
not surprising that significantly more upper classmen reported having received training 
on Tufts electronic research tools. 
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NewsLink vs. Lexis-Nexis 



Specific examination of students’ perceptions of the quality of two distinct web- 
based products revealed that students were generally satisfied with these on-line 
electronic news services and would recommend them to other students. Over three- 
quarters of the respondents indicated that they would recommend these products to other 
students (NewsLink 75.6% vs. Lexis-Nexis 79.1%). 

Usage of both NewsLink and Lexis-Nexis among students in the classes where 
faculty had purposefully introduced on-line electronic new services was surprisingly low. 
Approximately 41 percent of the students in these five classes indicated that they had 
used NewsLink or Lexis-Nexis. This comptires with 34.7 percent of graduating seniors 
who indicated they had utilized these products. Approximately 21 percent of the 1998 
seniors indicated that they were “not familitir” with these products, which suggests that 
“lack of awareness” is not the primary cause for limited use among students. 

Frequency of use varied between the two products. Students reported using Lexis- 
Nexis more often. Approximately 46 percent of the students used Lexis-Nexis more that 
once during the semester as compared to only 37.4 percent reporting the use of NewsLink 
more than once a semester. 

Access to the on-line services was reliable and there appetired to be very little 
difference between the two products. The majority of respondents found access to the 
NewsLink site to be sufficiently fast at all times of the day, with the overwhelming 
majority finding it sufficiently fast in the morning (96.8%) and at night (84.6%). This 
was equally true of Lexis-Nexis. The majority of students (67.2%) tended to access 
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NewsLink from their residence halls. In contrast, more students reported (36.7% vs. 

23%) accessing Lexis-Nexis from Tisch Library. 

It appears from the student perspective that both products provide ready access to 
recent and relevant news articles. Most (78.6%) of the students who used NewsLink to 
find articles for a class assignment or project were able to find a sufficient number of 
articles to support their research. This was also true for students (76.9%) who used 
Lexis-Nexis. 

Students found both products fairly easy to use. In fact, the vast majority (88.1%) 
found it "very" or "moderately easy" to find pertinent articles on NewsLink. This was 
also true for the Lexis-Nexis product. While most found it easy to find articles in their 
subject area, it is important to look at the difficulties students reported. For both 
NewsLink and Lexis-Nexis, several students stated difficulties in narrowing their search 
and were frustrated with having to sift through hundreds of irrelevant articles. A few 
respondents encountered technical problems, including having the programs crash on 
their home computers and not being able to print from campus computer labs. 

At the close of the spring 1998 semester, five faculty members were interviewed. 
The interview protocol was quite extensive. Faculty were asked to provide information 
regarding their satisfaction with the specific web-based tools, a description of how the 
tool was integrated into their curriculum, their perceptions of student use, and an 
assessment of how such tools could be made available to other faculty members in their 
department. 

Since not all faculty were familiar with both products, it is difficult to provide a 
comprehensive comparative evaluation. Of the three faculty members who had used both 
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products, it was clear that they preferred Lexis-Nexis. The primary concern with the 
NewsLink product was the organization and categorization of subject matter. News 
articles are grouped in ways that do not correspond with the current organizational 
stmcture of academic courses. In order to be a useful tool, faculty suggested that the 
categories need to be modified so that they more closely match standard divisions within 
each academic discipline. Faculty members in several disciplines (economics, sociology, 
and psychology) articulated this concern. In addition, there is no feature available that 
allows the users to set up a “personal keyword search” and receive only relevant articles 
on a regular basis. 

Faculty seemed more likely to use these on-line electronic news services for 
research purposes and to enhance the currency of their curriculum than as a teaching aid. 
In most cases, students were encouraged to examine these resources and use them to 
provide background information and direction for additional exploration. 

The five faculty members concurred that students were generally receptive to 
using these on-line electronic resources. However, they were quick to point out that 
students need at least mdimentary training in keyword search techniques for web-based 
research tools. 

METHODS OF NEWS ACQUISITION 

Members of the Class of 1998 and students enrolled in the NewsLink-cnhanc&d 
courses were asked a variety of questions to determine their preference for acquiring 
current events information, which medium they actually use for this purpose, as well as 
which newspapers they read on a regular basis. 
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Our analysis revealed that most respondents read a newspaper on a regular basis. 
Responses varied between the two surveyed groups, approximately 77 percent of 
graduating seniors reported that they do read newspapers, compared with 92 percent of 
the students in NewsLink-enhanced courses. Both groups most frequently read the Boston 
Globe, the New York Times, and the Tufts Daily. 



Medium Used to Learn about Current Events 




thcWWW 



j B Newslink Survey □ Senior Survey 



More students rely on television as their source of news. When given a choice, 
most students in NewsLink-enhanced classes prefer television to keep up with current 
events. Among the students who were enrolled in the NewsLink-enhanced classes, 79.3 
percent indicated that they relied on television as a news source, while 63 percent 
indicated that they used the newspaper. Similarly, 79.7 percent of seniors indicated that 
they relied on television as a news source, while 60.3 percent indicated that they used the 
newspaper. When asked which they preferred, 53 percent of students in NewsLink- 
enhanced classes identified “watching news on television” as the preferred source. 
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Graduating seniors were more split on their preferred medium for current events, with 
slightly more preferring hardcopies of newspapers (41.2%) than television (40.8%). 



Preferred Medium to Learn about Current Events 




Television Newspapers Newspapers on Radio Other: 

the WWW 



[ M New^ nk Survey □ Senior Survey j 

Examination of preferences by gender revealed significant differences. In both 
populations, a higher percentage of women than men relied on television as a source of 
current events. Conversely a higher proportion of men than women read hard copies of 
newspapers. In fact, almost 50 percent of the women indicated that their preferred source 



Preferred Medium for Current Events 
Men vs. Women 




the WWW 

'■Men □ Women! 
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of current events information was television as compared to only 34 percent of the men 
with a similar preference. A higher proportion of men (44%) identified hard copy 
newspaper as their preferred source. 

PROBLEMS ENCOUNTERED 

In the course of this evaluation, we encountered severed difficulties that seemed to 
stem from a lack of central control over academic and research technologies on campus. 
This problem is probably not unique to Tufts, and has potentied implications for other 
institutioned researchers undertaking evaluations of internet-based research and teaching 
tools. At Tufts, there were no university-wide procedures for the review, purchase, 
marketing and technical support of new electronic research and teaching tools. 

Depending on perceived potential users and uses, electronic resources may be purchased, 
and therefore marketed and supported, by different departments or schools. 

The NewsLink product in particular was problematic because of its multiple 
applications: for news acquisition, for research, or as a teaching aid. This created 
confusion over which department should be responsible for supporting and promoting the 
product. Amidst all this confusion, as institutional researchers, it was our responsibility 
to fulfill a contractual obligation to evaluate this product. However, because marketing 
for NewsLink was virtually non-existent, it was difficult to find a sufficiently large 
sample of current users who would be able to evaluate the product in terms of ease of use, 
access, reliability and desirability. 

The current structure of our Internet servers also created problems in collecting 
data that could be helpful in measuring frequency of use. Prentice Hedl provided us with 
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monthly data on the number of requests and bytes sent on various Tufts subdomains. 
These subdomains represent the different web servers on campus, some serving a single 
academic department or computer lab, others serving the bulk of administrative, student 
and faculty units. On the frequency reports, the names and number of subdomains 
reporting activity differed each month, preventing cross comparisons. In addition, the 
bulk of activity occurred on the two servers that serve all three constituencies, making it 
impossible to discern whether students, faculty or staff accessed NewsLink most 
frequently. 

SUMMARY 



Overall, students who used NewsLink and Lexis-Nexis were generally satisfied 
with these on-line electronic news services and would recommend them to other students 
Respondents were generally veiy satisfied with ease of access and the ability to find 
pertinent articles. However, less than 50 percent of respondents indicated that they had 
used the products in the past year. Thus, it appears that while students are frequently at a 
computer and on the Internet for email or research, they generally do not choose to use 
the Internet to get their current events information. Rather, Tufts students rely on 
television and newspapers as their sources of news information. Moreover, students 
prefer television to newspapers. 

Given the rapid increase in the utilization of the WWW, it is quite likely that this 
is the wave of the future. In fact, students may tdways prefer television as a source of 
news; however, with proper training and promotion, there is the possibility of increased 
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utilization of such web-based research tools. For these tools to be embraced, both 
students and faculty will need to use the products and determine whether they will 
enhance their research. 

It is clear that student and faculty use of electronic resources is not a passing 
phenomenon. The demand for resources to support these new technologies and sources 
of information, in all likelihood, will dramatically increase in the future. Thus, it is 
critical that institutions begin to examine both usage and effectiveness. The next step at 
Tufts will be to develop a research plan and protocol that will address how to 
systematically evaluate utilization of resources, student and faculty preferences, effective 
integration of technology in the classroom, and demand for training. 
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In the competitive market of student recruitment, college admissions offices are 
experimenting with the use of predictive models to increase the effectiveness of their 
recruitment efforts. Regression analysis is used to estimate students’ probability of 
enrollment. Then different recruitment activities are directed at students with different 
enrollment probabilities. This paper argues that a typical use of predictive modeling is 
theoretically unsound and may therefore be operationally inefficient. To test this 
hypothesis and explore an alternative use of predictive modeling we designed and 
assessed an experimental recruitment program. The first-year results confirm our 
perspective and identify a valuable role for statistical modeling in recruitment 
management. 



THEORETICAL CONSIDERATIONS 

Predictive modeling is frequently used to identify the students most likely to apply or 
to enroll in a college or university so that admissions staff can concentrate their attention 
on these “hot prospects” in order to enroll more students. While this is an attractive 
approach it may not be an efficient one. 

Consider, for example, the hypothetical responses shown in T able 1 to a recruitment 
initiative such as a special mailing or invitation to a campus open house. After the 
intervention almost all the students in Group A enroll. It increases their average 
enrollment probability from 80% to 85% and adds 5 students to the entering class. In 
Group B, the students have only a 30% chance of enrolling after the recruitment 
intervention, but it increases their probability by 10% and enrollment by 10 students. 

The students in Group A are “hot prospects” in that they are likely to enroll, but devoting 
admissions office efforts to their recruitment diverts resources away from the target 



' This project was a collaborative effort between Institutional Research, Admissions and 
the Department of Economics. Essential contributions were made by Gigi Lamens and 
Manuel London who made the experimental use of predictive modeling an integral part 
of their recruitment plan, Paula Pelletier and Dave Taiclet who assisted with data 
management, and Mark Montgomery who provided statistical advice. 
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population on which they would have the greatest impact. 



Table 1. Hypothetical Recruitment Intervention 





Group A 


Group B 


Number of students 


100 


100 


Probability of enrollment w/o intervention 


80% 


20% 


Probability of enrollment with intervention 


85% 


30% 


Yield without intervention 


80 


20 


Yield with intervention 


85 


30 


Effect of intervention 


5 


10 



To be efficient, recruitment programs should be directed at prospective students 
wavering on the brink of an enrollment decision and most susceptible to the additional 
encouragement provided by recruitment efforts. Admissions resources should be targeted 
where they will cause the greatest increase in the probability of students’ enrolling, and 
those may or may not be the students with the highest probability of enrollment. In the 
language of economics, admissions office resources will be used efficiently if they are 
used where they have the greatest marginal impact. It is these “fence sitters” rather than 
the “hot prospects” that predictive modeling should help identify. 

Focusing on high-probability students may also be ineffective by directing attention 
away from the most institutionally desirable prospective students. It is likely that high- 
achieving students have more attractive alternative admissions offers to consider and 
therefore lower enrollment probabilities. Conversely, the "hot prospect" high-probability 
students are likely to have weaker academic credentials. 

Though theoretically sound, targeting recruitment programs to students susceptible 
to persuasion is problematic in practice. Neither admissions personnel nor researchers 
know the efficacy of different interventions or how prospective students’ response to 
recruitment efforts varies with their absolute enrollment likelihood. Predictive modeling 
can, however, be used to develop efficiently targeted recruitment programs. By 
providing estimates of students’ pre-intervention enrollment probabilities, it can support 
experiments to test the marginal impact of various recmitment initiatives. The rest of this 
paper describes a test of the value of this approach. This project began with the 
development of a predictive model, used the model to select samples of students on 
which to test an experimental recruitment initiative, and conducted the experiment to 
determine its differential effect on students with different enrollment probabilities. 



PREDICTION IN PRACTICE 

Developing a predictive model of students’ enrollment decisions requires selecting a 
study population, identifying variables likely to affect students' enrollment probabilities, 
developing a prediction procedure, and using the model to test the effects of experimental 
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recruitment activities. This section describes how we completed each of these steps to 
develop a better understanding of our university’s applicant pool and support targeted 
recruitment efforts. 



Population. The model we developed predicts the probability that a student offered 
admission to our university as a full-time freshman in the fall will enroll. The focus on 
admitted students distinguishes this project from another common application of 
predictive modeling. Models can also be used to assess the likelihood that students who 
inquire about admission will actually complete an application. We chose instead to focus 
on admitted students because increasing the enrollment yield from this pool is a priority 
for our Admissions Office. Moreover, far more information is available about admitted 
students, increasing the likelihood of accurate predictions. 

Predictive Variables. The selection of variables for a predictive model of students’ 
enrollment decision depends on a combination of theoretical considerations — regarding 
the student characteristics likely to affect enrollment in a specific college — and practical 
considerations — regarding the availability of data. Our model includes four kinds of 
variables: demographic, academic, geographic, and behavioral (Table 2). Some are likely 
to be important with any student population while others may be relatively specific to our 
campus. Statistically insignificant variables are included because our goal is accurate 
prediction, and retaining all the variables increases the model’s overall accuracy. 

Most of the predictive variables are significant at at least the 10% level. The highly 
significant variables (1% level) positively related to enrollment are high-yield high 
school average, high-yield SAT score, high-yield math SAT, high-yield verbal SAT, 
high-yield zip code, and open house attendance. Dummy variables indicating whether a 
student's high school average and SAT scores are within ranges that historically yield a 
high number of enrollees are used instead of the raw values of these variables because 
there is no reason to believe they are linearly related to the probability of enrollment. 
Categorical variables with large numbers of values — like the student’s high school and 
zip code — can only be included in the regression through a classification scheme 
reflecting the historical relationship between students’ geographic origins and enrollment. 

Variables significant at the 1% level and negatively related to enrollment include 
White or Hispanic ethnicity, US citizenship, regular admission status, early application 
and on-campus housing request. The model’s insignificant variables are age, gender, 
Asian ethnicity, having an intended major, having an intended science major, speaking 
English as a native language, low family income, and residing in a high-yield zip code. 

Additional variables could improve the model. For example, the behavioral 
variables — relatively early application and open house attendance — offer very limited 
information about how eager students are to attend our university, and we hope to add 
other indicators in the future. 
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Table 2. Variables in the Logistic Regression Model 



Variable 


Coefficient 


Description 


DEMOGRAPHIC VARIABLES 




Age 


-0.2285 


college age: 17-19 yetu'S old 


Gender 


0.0448 


male = 1 , female = 0 


White 


-0.3770*** 


ethnicity is White 


Asian 


-0.0777 


ethnicity is Asian- American 


Hispanic 


-0.4139*** 


ethnicity is Hispanic-American 


Black 


-0.2705* 


ethnicity is African-American 


Citizen 


-0.7161*** 


United States citizen 


Permanent resident 


-0.4206* 


US permanent resident 


English 


-0.1056 


English is native language 


High income 


-0.1794** 


high self-reported family income (>$75,000) 


Low income 


0.0919 


low self-reported family (<$39,000) 


Status 


-0.4339*** 


l=regular admission, 0=special 


ACADEMIC VARIABLES 




HY HS average 


0.2389*** 


high-yield high school average 


HS missing+ 


1.0382*** 


missing high school average 


HYSAT 


0.3857*** 


high-yield SAT combined score 


SAT missing+ 


-0.9218*** 


missing SAT score 


HY math 


0.3048*** 


high-yield SAT Math score 


HY verbal 


0.2433*** 


high-yield SAT Verbal score 


Major 


-0.0718 


application indicated an airea of interest 


Science major 


0.1224 


application indicated science interest 


GEOGRAPHIC VARIABLES 




HYHS 


0.3929*** 


high-yield high school 


HYZip 


0.0902 


high-yield zip code 


NYC/LI 


0.4418*** 


lives in NYC or Long Island 


BEHAVIORAL VARIABLES 




Applied early 


-0.3872*** 


applied before December 1 


Open house 


0.9706*** 


attended an open house 


Campus housing 


-0.7676*** 


requested on-campus housing 


CONSTANT 


0.0594 





* significantly different from zero at the 10% significance level. 
** significantly different from zero at the 5% significance level. 



*** significantly different from zero at the 1% significance level. 

+ These variables permit cases with missing data to be included in the model. Actual 
scores are used where available, and an alternative coefficient is eissigned to missing data. 
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The model’s accuracy would almost certainly be improved by financial aid data such 
as the percentage of a student’s financial aid need met, the types of aid offered, and the 
date on which the financial aid offer was mailed. However, including these variables 
would limit the model’s usefulness because financial aid information does not become 
available until relatively late in the recruitment process. A model including financial aid 
measures could not be used to identify students' enrollment probabilities early enough to 
allow time for targeted recruitment efforts, and we chose not to include them. 

Prediction procedures. We used logistic regression to predict students' enrollment 
probabilities. The theory is simple. Consider, for example, a simplified case in which 
the only thing known about students is whether or not they are female. If in one year 100 
female students are admitted and 60 enroll, the probability of a female student's enrolling 
is 60%. Hence we can predict that the enrollment probability of a female student 
admitted in subsequent years will also be 60%. Logistic regression merely permits a 
large number of student characteristics to be incorporated in this t)q)e of calculation. 

Three years of data are required to develop the predictive model, test its stability, and 
use it to analyze recruitment initiatives. We used 1996 data to develop the model by 
estimating the values corresponding to 60% in the simplified example. Based on those 
values we predicted the enrollment decisions of students matriculating in fall 1997 and 
compared students' predicted and actual behavior to evaluate the model's accuracy. We 
then used the model to predict the enrollment behavior of students entering in fall 1998, 
with and without an experimental recruitment initiative. 

Prediction results. For logistic regression there is no simple statistic measuring the 
accuracy of a model comparable to the R-square statistic computed for linear regressions. 
Instead the model’s predictive power can be assessed by measuring goodness-of-fit 
through classification tables that compare results predicted by the model with students’ 
actual enrollment decisions. 

In order to draw this comparison it is necessary to decide what counts as a 
“prediction of enroll” since the regression predicts enrollment probabilities as a 
continuous variable. A cut-off probability level must be selected, above which a student 
is counted as being predicted to enroll and below which a student is counted as predicted 
not to enroll. The selection of this cut-off value is somewhat arbitrary. We used 0.30 
because about 30% of our admitted students enroll and because we prefer the way 
predictions using this cut-off compare with students' actual enrollment decisions. Lower 
cut-off points result in lower overall accuracy while higher cut-off points substantially 
underestimate the number of students who enroll. 
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Table 3 shows the results of using the model estimated using fall 1996 data to predict 
fall 1997 enrollment decisions. 

Table 3. Predicted and Actual Enrollment 





PREDICTION 




ACTUAL 


Predicted 


Predicted 




BEHAVIOR 


to enroll 


not to enroll 


TOTAL 


Enrolled 


1,219 


917 


2,136 




16% 


12% 


27% 


Did not enroll 


1,450 


4,223 


5,673 




19% 


54% 


73% 


TOTAL 


2,669 

34% 


5,140 

66% 


7,809 

100% 



The upper left and lower right quadrants represent correct predictions by the model: 
1,219 students (16% of the total) were predicted to enroll and actually enrolled, while 
4,223 students (54% of the total) were predicted not to enroll and did not enroll. The 
model is accurate 70% of the time, which is a significant improvement over the 
prediction possible without the regression model. An uninformed projection would 
predict enrollment correctly 27% of the time, since 27% of all admitted students actually 
enrolled. 

Table 4 offers further assurance that the model accurately assigns probabilities to 
admitted students by confirming that the actual enrollment decisions of students in 
different predicted probability ranges corresponds to the prediction. For example, the 
first row shows that 9% of the students with predicted enrollment probabilities between 
0% and 10% actually enrolled. 
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Table 4. Predicted and Actual Enrollment 
by Probability Range 



predicted 

enrollment 

probability 


percent 

enrolled 


number of 
students 


0%-10% 


9% 


1,495 


10%-20% 


17% 


2,219 


20%-30% 


28% 


1,426 


30%-40% 


39% 


1,051 


40%-50% 


46% 


768 


50%-60% 


51% 


450 


60%-70% 


58% 


270 


70%-80% 


57% 


103 


80%-90% 


57% 


23 


90%-100% 


0% 


1 


TOTAL 




7,806 



In ranges up to 60%, the percentage of students who actually enrolled falls within the 
predicted probability range. Above that there is a discrepancy between the predicted 
probability and the actual percentage who enrolled, but there are not many students in 
those ranges and more than 50% of the students in each range actually enrolled, double 
the overall enrollment percentage. The estimated model fits the data quite well. 

These results confirm the importance to our university of recruitment initiatives 
targeting students with low predicted enrollment probabilities. Most of both the admitted 
students and the students who actually enroll have low enrollment probabilities. 

Other institutions’ admissions pools may display different patterns. Our large 
number of admitted students with low enrollment probabilities may, for instance, be 
attributable to a systemwide application system that makes it easy for students to apply to 
several campuses. The distribution is institutionally important, however, because 
knowing its shape has helped the Admissions Office better understand its target 
population. 

The analysis of admitted students by enrollment probability also confirms the 
hypothesis that targeting recruitment efforts to the students most likely to enroll does not 
focus attention on those with the strongest academic credentials. Enrollment probability 
is inversely related to average SAT score (Table 5). 
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Table 5. Average SAT Score 
by Probability Range 



predicted 

enrollment 

probability 


Average 

SAT 

score 


number 

of 

students 


0%-10% 


1250 


1,495 


10%-20% 


1217 


2,219 


20%-30% 


1169 


1,426 


30%-40% 


1120 


1,051 


40%-50% 


1058 


768 


50%-60% 


1034 


450 


60%-70% 


1012 


270 


70%-80% 


1020 


103 


80%-90% 


976 


23 


90%-100% 


- 


1 


TOTAL 




7,806 



In this population of admitted students, targeting hot prospects does not direct 
recruitment efforts to high-achieving students. Effective recruitment requires attention to 
low-probability students. To investigate whether recruitment initiatives aimed at students 
with low enrollment probabilities would also be efficient, we designed a recruitment 
experiment. 

A RECRUITMENT EXPERIMENT 

Experimental design. With enrollment growth an institutional goal, our Admissions 
Office was interested in identifying recruitment efforts that would increase freshman 
enrollment by increasing the relatively low percentage of admitted students who enroll. 
Specifically, Admissions wished to test the efficacy of increased contact with admitted 
students and their parents through activities requiring modest staff effort so that, if 
successful, those activities could be expanded to a large group of students. 

To carry out this test, we designed an experiment to compare the enrollment 
decisions of students to whom the new program was directed — the experimental group — 
and students not affected by it — the control group. The experiment focused on students 
with enrollment probabilities between 30% and 60% — the middle of the distribution — 
because the experimental recruitment initiative was designed for use with a large number 
of prospective students. 

The experimental and control groups are shown in Table 6. The experimental group 
initially included 200 students, the largest number to which the Admissions staff felt they 
could devote additional attention, divided into equal-size experimental groups to facilitate 
comparisons among the three probability ranges. This experimental group was 
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subsequently expanded to include half the students in the highest predicted probability 
ranges. The control groups were all students with comparable enrollment probabilities 
who had been admitted at the time the samples were drawn. 

Table 6. Experimental Groups 



Probability 

range 


Experimental 

group 


Control 

group 


30-40% 


67 


447 


40-50% 


67 


178 


50-60% 


67 


64 


>60% 


125 


130 



Students in the experimental group received an additional invitation to visit the 
campus, and their parents received two special mailings. Most of these students also 
received expedited financial aid packaging, and as many as could be reached were 
contacted in a financial aid telethon. The experiment was not perfectly controlled in that 
it included several different interventions, some of which did not include every student in 
the experimental group. It was, however, sufficient to provide initial evidence of the 
program’s efficacy. 

Experimental results. The experimental program increased enrollment in the groups 
with relatively low enrollment probabilities. Of the students with enrollment 
probabilities between 30% and 50%, 42% of the experimental group enrolled compared 
to only 33% in the control group (Table 7), and this difference is statistically significant. 



Table 7. Effects of the Recruitment Experiment 



Probability of 
enrollment 




Enrolled 


Not 

enrolled 


Number 


30-50% 


Experiment 


42% 


58% 


134 


p = .04 


Control 


33% 


67% 


625 


50-60% 


Experiment 


33% 


67% 


67 


p=.05 


Control 


50% 


50% 


64 


60-90% 


Experiment 


49% 


51% 


125 


p=.24 


Control 


56% 


44% 


130 



In the groups with higher enrollment probabilities, the experimental program did not 
increase enrollment. Additional attention appears to have had no effect on students with 
the highest enrollment probabilities — the 60% to 90% group. In this probability range 
fewer students in the experimental group enrolled, but the difference between the 
experimental and control group is not statistically significant. In 50-60% range fewer 
students in the experimental group enrolled, and the difference is statistically significant, an 
unexpected result that may be a product of small sample size. 




159 



159 



While the accuracy and reliability of these results is limited by the small sample and 
imperfectly controlled experiment, they strongly suggest that students with relatively low 
enrollment probabilities are more susceptible to increased recmitment efforts. We plan to 
replicate and refine the experiment in future years with the hope of confirming this 
conclusion and determining the contribution of the different elements included in the 
experimental recruitment program. 

CONCLUSION 

The recruitment experiment indicates that to support enrollment growth our Admissions 
Office should include students with relatively low enrollment probabilities among its 
recruitment targets. A modest increase in the attention these students receive appears to 
have a significant effect on their behavior. Extending the experimental recruitment 
program to all students with enrollment probabilities between 30% and 50% would have 
increased the freshman class by about 70 students or 3%. 

These results confirm the importance of the insight on which this project was based. 

It appears that recruitment efforts should not focus on “hot prospects,” though further 
research with larger samples, more strictly controlled experiments, and different 
admissions pools is needed to verify this conclusion. Concentrating recmitment efforts 
on the “hot prospect” students appears to be inefficient — by diverting resources away 
from the population on which they have most effect — and ineffective — by focusing on 
students who are not the strongest candidates in the admissions pool. 

The experiment also demonstrates a more general point: recmitment initiatives are 
relatively easy to assess. Compared to other assessment targets the outcome to be 
measured is simple: students either do or do not enroll. By providing a baseline 
prediction of students' behavior, predictive modeling can be a valuable research tool in an 
admissions office willing to experiment and assess the results of different recmitment 
activities. 

Technical feasibility is not, however, the only issue in implementing assessment- 
based recmitment management. This approach makes significant demands on admissions 
staff who must have good data in a usable form and be willing to take some unusual 
risks. An experimental approach requires devoting resources to the recmitment of 
students who are unlikely to enroll. While a controlled experiment is in progress it also 
requires excluding some students from recmitment efforts that could increase enrollment. 
These are difficult actions for admissions staff under pressure to meet enrollment targets. 

In this context the research orientation of institutional research staff can provide 
encouragement while their technical expertise supports innovation. A project such as this 
requires active collaboration between institutional research and admissions to insure that 
predictive modeling is more than an academic exercise, but it can be a very fmitful 
collaboration. 
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ABSTRACT^ 

Do administrators in public higher education experience different levels of job 
satisfaction than their counterparts in the private sector ? Drawing upon the management, 
higher education, and public administration literature, this study examines a 
comprehensive array of national data on university characteristics, state characteristics 
and administrative satisfaction. Such research is important because of the connection in 
the literature between levels of satisfaction and employee productivity and managerial 
turnover. The findings suggest that the hypothesized public/private differences are 
limited to the sub-components of satisfaction reflecting extrinsic rewards and 
relationships with others. In both sectors, job satisfaction is most consistently linked to 
work environments characterized by teamwork and low levels of interpersonal conflict. 



The authors acknowledge with appreciation the earlier data collection and database 
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Comparing Administrative Satisfaction in 
Public and Private Higher Education 



The Research and Policy Problem 

Much has been written about managerial satisfaction in business and public 
administration, but most studies in higher education have examined the satisfaction levels 
of faculty rather than administrators (Austin & Gamson 1983, Gmelch, Lovrich, and 
Wilke 1984, Cotton and Tuttle 1986, Smart 1990, Olsen 1993, Hagedom 1994). The few 
studies of administrative satisfaction in higher education focus primarily on 
understanding the nature and level of satisfaction, rather than on examining the factors 
producing satisfaction and the subsequent connections to important outcomes such as 
turnover and productivity found in the management literature (Solomon & Tierney 1977, 
Smart and Morstain, 1975, Blix and Lee 1991, Glick 1992). 

One important claim in higher education is the connection between autonomy and 
quality. Previous attempts to empirically measure the relationship between quality and 
autonomy at the institution level haye proved inconclusive (Volkwein 1986, 1987, 1989; 
Volkwein & Malik 1996, 1997). However, the literature on job satisfaction leads one to 
expect that autonomy may indirectly effect institutional quality through gains in 
productivity that result from job satisfaction (Vroom 1964, Porter and Lawler 1968). A 
previous study of public universities reveals little direct relationship between campus 
autonomy and administrative satisfaction in public universities (Volkwein, Malik and 
Napierski-Prancl, 1998). While their study did not examine private universities as 
perhaps the most extreme examples of autonomy, it found a consistent connection 
between every measure of administrative satisfaction and the human relations aspect of 
university administration. 

These findings from the higher education literature are consistent with evidence 
from public administration research that found public professionals deriving their job 
satisfaction primarily from the social aspects of their jobs and only secondarily, from the 
work itself (Emmert and Taher, 1992). Additional public administration literature also 
suggests that job satisfaction varies in significant ways between the pubic and private 
sector (DeSantis and Durst, 1996; Steel and Warner, 1990). In this regard, it is both 
reasonable and important to explore the nature and levels of administrative satisfaction in 
public and private universities in order to understand any important differences that may 
exist. 

Purpose of the Study and Conceptual Frameworks 

Do administrators in public and private higher education experience different 
levels of satisfaction ? Are the influences on administrative job satisfaction similar or 
different in public versus private universities? This research examines the perceived 
work environments and individual characteristics of administrative managers in both 
public and private higher education. In addition to survey data on perceived work 
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environments, we have also incorporated university and state characteristics consistent 
with a variety of theoretical perspectives from the research literature: organizational 
theory, structural/functional perspectives, the literature on university autonomy, and 
theories of employee satisfaction. 

Job Satisfaction 

At the core of this study is the job satisfaction literature. There is general 
agreement in the literature that job satisfaction is multi-dimensional. Herzberg’s Two 
Factor Theory (1966) draws our attention on the one hand to intrinsic job content factors 
(such as feelings of accomplishment, recognition, and autonomy), and on the other hand 
to extrinsic job context factors (such as pay, security, and physical working conditions). 
Several studies have examined the intrinsic and extrinsic dimensions of job satisfaction in 
higher education (Olsen, 1993; Austin and Gamson 1993; Hackman and Lawler, 1971, 
Kalleberg 1977, Hagedom 1994). 

In addition to agreement in the literature that job satisfaction is multi-dimensional, 
most studies conclude that satisfaction is influenced by a complex array of personal and 
situational circumstances (Austin & Gamson, 1983; Hoppock, 1977; Mumford, 1972; 
Brace and Blackburn, 1992). Research has shown that several work related variables 
exert positive and significant influences on administrative satisfaction — a supportive 
organizational culture, teamwork, relationships with colleagues and superiors, worker 
autonomy, and self-fulfillment (Berwick 1992, Bensimon & Newman 1991, Austin & 
Gamson 1983, Boone 1987, Lawler 1986, Rigg 1992, Volkwein, Malik and Napierski- 
Prancl, 1998). Most management experts to argue in favor of worker autonomy and 
organizational flexibility (< biblio >) use Maslow’s hierarchy of needs. Job and 
workload stress exert negative influences on satisfaction and are almost always included 
in studies of job satisfaction (Blau 1981, Blix & Lee 1991, Glick 1992, Olsen 1993, 
Hagedom 1996, Volkwein, Malik and Napierski-Prancl, 1998) 



Organization and Environment Perspectives 

Perspectives from organization theory emphasize the importance of the 
organization’s structure and its environment (Hall, 1995; Lawrence and Lorsch, 1967; 
Aldrich, 1979; Pfeffer and Salancik, 1978). Scholars often divide an organization’s 
environment into economic, political, social, and technological dimensions. Additional 
research, specifically from studies of colleges and universities, have demonstrated that 
campus mission, size, wealth, complexity, and selectivity exert significant influences 
(ranging from small to large) on a variety of college outcomes (Pascarella and Terenzini, 
1991; Austin & Gamson 1983; Hall 1995). The organizational and environmental 
literature generally leads us to expect that an array of campus and state characteristics 
may exert significant influences on administrative satisfaction. 
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Individual Characteristics 



Consistent with research in other organizations, studies of managers in colleges 
and universities suggest that a variety of personal and organizational variables exert 
potential influences on their job satisfaction. Among these personal characteristics are 
physical and mental health, age (Austin, 1985; Lee & Wilbur, 1985; Solomon & Tierney, 
1977), sex (Austin 1985; Hagedom 1996), level of education (Martin and Shehan 1989), 
length of service (Bamundo & Kopelman 1980), administrative rank (Austin & Gamson 
1983), and administrative area (Click, 1992). 

Research Methods 

Guided by these various perspectives and concentrating on variables potentially 
important to job satisfaction, we created the analytical database for this study drawing 
information from a wide range of different sources. This study employs the same 
methodology as an earlier investigation that examined administrative satisfaction and the 
regulatory climate at public universities (Volkwein, Malik, & Napierski-Prancl, 1998). 
The previous investigation collected survey responses simultaneously from both public 
and private university administrators at cooperating campuses. We added data reflecting 
campus organizational and environmental characteristics and then engaged in data 
reduction techniques using principle components analysis and scale building techniques. 
The resulting variables and scales form the basis for examining the nature of 
administrative satisfaction in public and private universities. 



Target Population, Data Sources, & Variable Summary 

Table 1 summarizes the nature of the variables used in this study. Campus 
characteristics reflecting organizational mission, size, wealth, quality, and complexity are 
based upon the factor analytic and scale building procedures described in Volkwein and 
Malik, 1997. Sources for the data include NCES/IPEDS, the National Research Council 
study of doctoral programs(1995), the Graham and Diamond national study (1996), and 
the guidebook information contained in Barron’s and US News. 

Our scale of autonomy and flexibility measures the nature of the external 
environment at each institution. For public institutions, we use the three-category 
(high/medium/low) scales developed by Volkwein and Malik (1997). For the separate 
administrative and academic autonomy dimensions, each campus is classified as high 
(one standard deviation above the mean), low (one standard deviation below the mean), 
or medium. This forms the first three categories of a four-category continuum. All 
private universities are classified as falling in the fourth (or highest) autonomy category. 

The respondent characteristics (age, ethnicity, sex, rank, etc.), as well as their 
reported working conditions, personal stress, and satisfaction levels, are extracted from 
our survey that was administered by cooperating campus officials to a population of 
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managers at public and private universities. This survey was administered to 12 
managers (ranging from vice presidents to directors) on each campus. This survey 
contains 7 questions about the respondent’s background, and 44 items assessing their 
satisfaction, stress, and working conditions. Using follow-up procedures that guaranteed 
respondent anonymity, we eventually received an 80% response rate -- about 1200 
satisfaction surveys from 120 cooperating public and private universities. 

The survey assesses the respondents’ work climate and sources of stress on a 
series of survey items with 5-point response scales. The resulting measures are consistent 
with those produced by Volkwein et al.(1998), and are summarized in Table 1. Finally, 
the survey contains 25 satisfaction questions where responses are solicited on a 5-point 
Likert-type scale from very satisfied (5) to very dissatisfied (1). Congruent with the 
procedures employed by Volkwein, et al.(1998), our principal components and data 
reduction techniques produced four scales in addition to the single “overall satisfaction” 
measure: intrinsic satisfaction, extrinsic satisfaction, satisfaction with working 
conditions, and satisfaction with colleagues and other people one works with. The alpha 
reliabilities on these four multi-item scales are shown in Table 1 for the two populations. 
They range from .74 to .90. These four scales plus the single “overall” item are the 
dependent measures in this study. While the original scales were constructed using a 
public university population, the alpha reliabilities of the work climate, stress, and 
satisfaction scales from private university administrators are in most cases even stronger 
than those in public universities. More information about the psychometric 
characteristics of these measures can be obtained from the first author. 

Tables 2a and 2b show the number of respondent’s by functional area and 
rank, as well as the number of respondent’s from public versus private 
universities. As described in Volkwein et al.(1998), we targeted these managers 
for the study not because they necessarily represent all university administrators, 
but because they occupy particular job titles which are potentially influenced by 
the external environment. These 120 universities are the ones that agreed to 
participate in the study and they represent approximately half of all the nation’s 
doctoral granting universities. The totals across the tables are not completely 
congruent because not every respondent completed every item on the survey. The 
greatest representation of administrative respondents is from the “Director” rank 
and the “Business and Finance” functional area. Additionally, two-thirds of the 
respondents are between ages 45 and 60 and 75% are male. Most have a doctoral 
or masters degree. 

Analytic Procedures 

The study uses principal components analysis to collapse the survey items into a 
smaller number of scales that reflect the concepts in the literature and that are congruent 
with the previous study of public universities. The internal consistency of each scale is 
tested using Cronbach’s Alpha. Once the scales and individual measures were identified, 
we examined the data using descriptive statistics and compared the responses from those 
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in public versus private universities. We identified five dimensions of administrative job 
satisfaction. In order to assess any differences in satisfaction between public and private 
administrators, separate OLS stepwise regression equations were constructed for each of 
the five types of satisfaction for public and for private universities. This enabled us to 
examine and compare the influences on each satisfaction dimension identified in the 
research literature and empirically derived from our survey data. 



Results 



Do administrators in public and private universities experience different levels of 
satisfaction? Table 3 compares the public and private university responses on each of the 
five dimensions of administrative job satisfaction. On four of the five satisfaction 
dimensions, there are no significant differences in the mean responses and standard 
deviations of the public and private sector respondents. Both groups are most satisfied 
with the intrinsic rewards of their positions (3.9 to 4.0 on a 5-point scale). Both groups 
are least satisfied with their extrinsic rewards and work conditions (3.2 to 3.5 on a 5-point 
scale). The largest and only significant difference between the two groups of 
administrators occurs on the extrinsic rewards dimension (salary, benefits, promotion), 
where the private university group is significantly more satisfied than the public. Thus, 
we find differences between the two groups that are smaller than one might expect from 
reading the literature. 

Are the influences on administrative job satisfaction similar or different in public 
versus private universities? Table 4 shows the results of the multivariate analysis. For 
each of the five dependent measures the beta weights for the public and private regression 
models are displayed side-by-side. 

The first two columns show the beta weights from the OLS regression models 
using overall satisfaction as the dependent measure. In both the public and the private 
models, the largest influence on overall satisfaction is interpersonal conflict, followed by 
teamwork. In other words, controlling for all other variables, an absence of interpersond 
conflict and an atmosphere of teamwork account for most of the explained variance in 
overall satisfaction among both populations. The private university model is especially 
robust with an R-square of .39. In the public university model(R-square=.27), two other 
variables exert a lesser but significant negative influence on satisfaction - a controlled 
work environment and workload/time pressure. 

The second pair of columns in Table 4 presents the results of the intrinsic 
satisfaction models where the public university model obtained a total R-square of .3 1 
and the private university model obtained an R-square of .40. In both columns the largest 
beta weight is the negative influence of interpersonal conflict. In the public university 
model, intrinsic satisfaction is associated additionally with teamwork(.26) controlled 
work environment (-.17), age(-.ll), and rank(-.09). This indicates that respondents are 
more satisfied if they are in work situations without interpersonal conflict, if they are 
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involved in teams, if they occupy less controlled work environments, if they are younger, 
and if they have higher administrative rank. In the private university model, intrinsic 
satisfaction is additionally associated with rank(-.23) with teamwork(.21), and with 
female(-.18). Thus private university respondents are more satisfied if they are involved 
in teams, are male, and have a higher administrative rank. 

The third pair of colunms in Table 4 presents the results of the extrinsic 
satisfaction models where the public university model obtained a total R-square of .17 
and the private university model obtained an R-square of .23, results that are less robust 
than the other models in the table. The private university model is the least complex in 
that satisfaction with extrinsic rewards is significantly associated with only two variables: 
interpersonal conflict and rank. Among these respondents, the absence of interpersonal 
conflict(-.40) is almost twice as important as a high rank (-.26). In the public university 
model, extrinsic satisfaction is also associated with rank(-.17) and interpersonal conflict 
(-.16), but several other variables play a significant role. Among these are undergraduate 
quality(.12), having a medical complex(.lO), female(-.08), highest degree(-.08), being in 
academic affairs(.13), teamwork(.15), and campus funding(-.12). This indicates again 
that respondents are more satisfied if they are in work situations without interpersonal 
conflict, if they are involved in teams, and if they have higher administrative rank. 

The fourth pair of columns in Table 4 presents the results of the models for 
satisfaction with work conditions where the public university model obtained a total R- 
square of .42 and the private university model obtained an R-square of .54. These are the 
most robust models in our analysis. In the private university model, satisfaction with 
working conditions is most strongly associated with interpersonal conflict(-.44), followed 
by workload and time pressure(-.36) and teamwork(.19). The same variables are 
significant in the public university model with similar beta weights, but one other 
variable exerts a lesser but significant negative influence on satisfaction — campus 
funding. However, the bottom line here is that both public and private university 
respondents are more satisfied if they are free of interpersonal conflict and undue 
workload pressure, and are involved in teams. 

The fifth and last pair of columns in Table 4 presents the results of the satisfaction 
models for relationships with others. The public university model obtained a total R- 
square of .22 and the private university model obtained an R-square of .26. The private 
university model is the least complex in that satisfaction with the people one encounters 
on the job is significantly associated with only two variables: interpersonal conflict(-.40) 
and teamwork(.22). Among these respondents, the absence of interpersonal conflict is 
almost twice as important as a working atmosphere of teamwork. In the public university 
model, satisfaction is again associated most strongly with interpersonal conflict (-.29) and 
teamwork (.19), but several other variables play a significant role. Among these are 
campus size(-.17), faculty quality(.lO), undergraduate quality(.lO), having an agricultural 
college(.08), being in academic affairs(.13), and perceiving a climate of regulation(-.08). 



Discussion and Conclusions 



This is the first study in higher education to compare administrative satisfaction 
among public and private university managers. We collected survey data from 1200 
administrators ranging from Presidents and Vice Presidents to Directors and Assistants, 
and derived five dimensions of satisfaction. We found more similarities than differences 
between the two populations. There are no statistically significant differences between 
public and private university administrators in the levels of overall satisfaction, 
satisfaction with intrinsic rewards, satisfaction with working conditions, and satisfaction 
with the people they come in contact with in their work. Only in the area of extrinsic 
rewards do private university managers report significantly more satisfaction. Both 
groups on average are above the midpoint of the 5-point response scale on all five 
satisfaction dimensions, and both groups report being most satisfied with the intrinsic 
rewards of their jobs and least satisfied with the extrinsic rewards and work conditions. 

While public administration research suggests that there are significant 
differences between the public and private sectors with respect to levels of satisfaction, 
there is little evidence in this university population apart frorri pay and benefits that 
confirms this hypothesis. Even the satisfaction differences for extrinsic rewards (3.23 
versus 3.47 on a 5-point scale), while statistically significant, are hardly dramatic and do 
not translate into differences regarding overall satisfaction. 

In the multivariate analysis we examined the causes of satisfaction and 
dissatisfaction. Our analysis suggests that work place relationships and an atmosphere of 
teamwork are almost universally important contributors to every dimension of 
administrator satisfaction. This finding supports the scholarship that emphasizes 
teamwork and cooperative work arrangements (Bensimon and Neuman, 1993). It is also 
consistent with Hagedom’s recent research (1996) showing that interpersonal 
relationships positively influence Job satisfaction and also lessen Job-related stress. 

Other variables in the study are associated with specific dimensions of 
satisfaction. For example, holding a higher administrative rank is significantly associated 
with both intrinsic and extrinsic satisfaction in both populations. The pressures of 
workload and time significantly reduce satisfaction with working conditions in both 
populations. In public universities, campus size tends to reduce administrator satisfaction 
with the quality of their relationships with others in the work place, and undergraduate 
quality tends to promote satisfaction in two of the models. 

In general, however, few of the campus characteristics seem to influence 
administrative satisfaction for either population and neither of the two autonomy 
measures exerts an influence anywhere in the study. This finding contradicts the 
literature on campus autonomy, but is consistent with the previous study by Volkwein, 
Malik, and Napierski -Panel (1998). 

This study does not explore the relationship between Job satisfaction and other 
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important outcomes, such as turnover and productivity. However, understanding the 
nature of job satisfaction in higher education and any significant differences across 
sectors is an important first step to exploring this question further. Additionally, ample 
literature on job satisfaction alerts managers in all sectors to the potential importance of 
job satisfaction. In this regard, higher education administrators and policy makers need 
to consider the implications for job satisfaction as various “reforms” and management 
techniques find their way into the work place. Clear and consistent evidence from this 
study and the previous one (Volkwein et al., 1998), indicate the importance of a work 
environment that is team oriented and free from interpersonal conflict. The sources of 
stress and conflict in this study include turnover, job security, interpersonal relations with 
supervisors and colleagues, and lack of personal respect. Efforts to “rightsize” and 
“restructure” higher education institutions without regard for the effect on work units or 
teams have potentially dire consequences for the levels of satisfaction, on all dimensions, 
experienced by administrators. 
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Introduction 

Underprepared college students are students with basic deficiencies in academic 
skills necessary for the satisfactory completion of college-level course work. As a major 
academic support mechanism, developmental education provided for such students has 
long become an important component of instructional activities at many American 
colleges and universities. This is especially so with public 2-year institutions, where 41 
percent of the freshmen enrolled in developmental courses nationwide in fall 1995 
(USDE, 1996b). At Maryland community colleges, this figure reached almost 60 percent 
during the 1994-1995 academic year (Maryland Higher Education Commission, 1996). 
Given the diminishing resources for higher education in the recent decades, the 
increasing demand for developmental education has evoked a controversy as to whether 
or not colleges and universities should end or limit remediation after high school (Ignash, 
1997). While this debate may well continue into the next millennium, institutional 
researchers at community colleges can at least be certain of one thing right now: With 
the expected 16 percent enrollment increase in higher education over the next ten years 
(USDE, 1996a) as well as the open-admissions policies of community colleges, the 
underprepared student population on our campuses is most likely to increase in the 
foreseeable future, and institutional researchers will more frequently find themselves 
involved in the evaluation of developmental education programs in response to 
accountability demands. 

Prince George’s Community College (PGCC) is a comprehensive community 
college in Prince George’s County, Maryland that enrolls approximately 12,000 credit 
students each fall and spring semester. Reflecting the demographic characteristics of the 
county, some 70 percent of our students are African American. To ensure that our new 
entrants are fully prepared for college studies, they are required to demonstrate their basic 
academic skill proficiencies in reading, writing, and mathematics when seeking credit 
course enrollment for the first time. Students can do this either through placement testing 
or through developmental course completion. Past records indicate that the percentage of 
underprepared students identified by placement testing has been consistently high. The 
Office of Institutional Research and Analysis (OIRA) at the college conducted a research 
in 1995 to investigate the four-year academic outcomes of the underprepared students in 
the fall 1990 freshmen cohort. The findings of the research subsequently helped the 
college administration take new initiatives to improve the academic progress rates of 
these students. This summer, a research project was carried out at OIRA to revisit the 
issue of underprepared students’ academic outcomes. By taking the academically 
deficient students in the fall 1994 freshmen cohort as a sample, the current study was 
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expected to identify the factors affecting their academic achievement so that measures 
could be taken to help these students really benefit from the democratic access to higher 
education in this nation. 



Literature Review 

Astin’s (1991, 1996) input-environment-outcome (1-E-O) model has provided an 
important conceptual framework for studying academic outcomes in higher education. In 
his terminology, inputs refers to the personal characteristics the student initially brings to 
the institution, including the level of talent the individual previously developed. 
Environment refers to the student’s actual educational and non-educational experiences at 
the institution that are associated with various programs, policies, faculty and peers. 
Outcomes refers to the “talents” that the institution seeks to cultivate in the student. By 
focusing on the change or growth in the student after being exposed to the environment, 
this model enables the faculty and administration to find the type of environmental 
conditions that may best facilitate the development of student talents. 

A presentation of academic assessment theories will be incomplete without the 
mention of models for studying student retention, mainly because these models invariably 
include academic outcomes as an important intervening construct affecting the student’s 
decision to persist or not. Tinto’s (1975, 1987) attrition model is probably the most 
popular theoretical framework whenever student retention becomes of research interest. 

In Tinto’s view, the student enters the institution with a spectrum of background 
attributes and high school experiences that lead to the individual’s educational goals and 
initial cormnitment to the institution. Together, these factors influence the way the 
student interacts with, and gradually integrates into the academic and social systems of 
the institution. The degree of academic integration and social integration in turn changes 
the student’s initial commitment to the institution, which eventually makes the individual 
persist or exit from the college. Largely based on the notion of person-environment fit, 
Tinto’s model is especially helpful in locating problems in the interwoven systems of 
student retention. 

Considering the unique characteristics of the growing number of nontraditional 
students on college campuses in recent decades. Bean and Metzner (1985) proposed a 
new student attrition model to explain the withdrawal decisions of these older, part-time, 
and commuter students. Partly derived from Tinto’s (1975) traditional student attrition 
model. Bean and Metzner’ s model states that the phenomenon of nontraditional student 
attrition can be accounted for by four sets of variables: the individual’s poor academic 
performance, intent to leave as a function of psychological outcomes and academic 
variables, background and defining variables, and environment variables. Noticeably, 
the two researchers omitted social integration as a primary component of the model based 
on the empirical data, and acknowledged the compensatory impact of environmental 
factors for the negative influence of the academic variables. Apparently, Bean and 
Metzner’ s model can better address the student retention issue at community colleges 
than Tinto’s model. 
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In the last ten years or so, a considerable number of studies have been carried out 
by individual institutions or government agencies to assess the academic outcomes of 
underprepared community college students (e.g., Haeuser, 1993; Maryland Higher 
Education Commission, 1996; Schoenecker, Bollman, & Evens,1996; Seybert & Soltz, 
1992; Sinclair Community College, 1995). While most of these studies were descriptive 
in nature, there emerged some inferential statistics-based research projects that utilized 
recognized outcomes assessment and student attrition models to varying degrees. 

Long and Amey (1993) applied Astin’s (1991) I-E-0 model to the study of 
student success with a sample of underprepared students at Johnson County Community 
College in Kansas. Using a one-way multivariate analysis of variance (MANOVA) with 
a follow-up discriminant analysis, the two researchers were able to identify two input 
variables (reading scores and reading placement level, and high school GPA), one 
environmental variable (number of first term credit hours), and two output variables 
(highest developmental English course completed and nondevelopmental GPA) that 
could best distinguish the successful and unsuccessful groups of these underprepared 
students. 

In an example of the application of Astin’s (1991) I-E-0 model as well as Tinto’s 
(1975, 1987) and Bean and Metzner’s (1985) student attrition models, Campbell and 
Blakey (1996) assessed the impact of early remediation on the persistence and/or 
performance of underprepared students at a midwestem, suburban community college. 
Results from a regression analysis indicated that cumulative GPA, number of remedial 
courses, early remediation, first year remedial course taking, and a degree-seeking intent 
all significantly impacted student persistence. It was also revealed that age, ethnicity, 
gender, and a degree-seeking intent were significant predictors of the academic 
performance of underprepared students. 

At PGCC, Boughan (1995) conducted a comprehensive analysis of the 
developmental placement and academic progress issue by tracking underprepared 
students in the fall 1990 entering cohort. In a related study, Boughan and Clagett (1995) 
examined the four-year academic achievement of the whole cohort by means of logistic 
regression analysis. Although not explicitly model-based, they incorporated into their 
research design previous findings in the model-related outcomes assessment literature. 
Findings form this study suggested that significant predictors of academic achievement 
of degree-seeking students, prepared and underprepared alike, included cumulative GPA, 
summer session attendance, curriculum change, good academic standing, average credit 
hour load, remediation completion, first-year average credit hour load, developmental 
course taking, no curriculum choice, first-year good academic standing, immediate entry 
from high school, enrollment in hi-tech or allied health programs, and under 21 years of 
age. In the sequel, Boughan (1997) used path analysis to model the academic 
performance of community college students, thus effectively verify the findings from the 
Boughan and Clagett (1995) study. 
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An inspection of the literature on the academic outcomes of underprepared 
community college students shows that cumulative GPA is probably the best predictor of 
the academic success of these students. It also shows that, despite the apparent 
theoretical and empirical gains form model-, and inferential statistics-based studies, such 
research endeavors are still insufficient in number, if we take into consideration the mere 
size of the target student population nationwide who would enjoy the benefit of any 
policy change at their institutions resulting from the findings of our studies. Therefore, 
for both our better understanding of the factors affecting academic outcomes of these 
students and any potential improvement we can bring to developmental education 
programs, institutional researchers at public 2-year colleges should devote more time and 
effort to the study of this assessment issue. 

Method 



Subjects 

The subjects for this study (N= 1,249) were all the fall 1994 degree-seeking first- 
time entrants at PGCC who took the placement testing, and were identified as 
academically deficient in at least one of the three areas: reading, writing, and 
mathematics. Following the PGCC student outcomes typology as presented in Boughan 
and Clagett (1995), subjects were defined as achievers if they had earned a degree or 
certificate from the college, or transferred to a senior college, or earned at least thirty 
credits with a cumulative GPA of 2.0 or above, by the end of summer session 1, 1998. 
The remaining subjects, whether they still enrolled at the college in spring 1998 or not, 
were all regarded as nonachievers. To assess their academic performance and outcomes 
between their first semester and summer session 1, 1998, the end-of-semester student 
records maintained by the college information systems for the corresponding semesters 
were reviewed, and the relevant information was extracted. 

Research Design and Data Analysis 

This study was designed under Astin’s (1991, 1996) I-E-O model. Although this 
model explores the three relationships among the model components: inputs and 
environment, environment and outputs, and inputs and outputs, the relationship between 
inputs and environment was not examined in this study for its lack of immediate research 
interest. To answer the research question as to whether there existed a nonlinear 
combination of input and environmental variables that could effectively determine the 
binary outcome status of the subjects, logistic regression analysis with likelihood-ratio- 
based forward stepwise selection was selected as the major statistical method for the 
study. SPSS for windows Release 8.0 was used for the analysis, which involved a total 
of thirty-one variables based on the literature on outcomes assessment and student 
attrition, with academic outcomes as the dependent variable. Table 1 summarizes the 
relevant information of these variables. There were 15 percent of missing values with the 
variable B90. They were replaced with their estimates by employing the linear trend at 
point method as implemented in SPSS. 



Table 1. 

Name and Type of Variables for Logistic Regression Analysis 

Input Variables 
AGE; Age (Continuous) 

SEX; Gender (Binary, male=0, female=l) 

RACE; Race/ethnicity (Binary, white=0, nonwhite=l) 

HSTYPE; High school type (Ordinal, PG private school=l, elite PG public school=2, 
other PG public shool=3, other school=4) 

B90; Social economic status (Ordinal, a 15-category scale based on Boughan & 

Diehl (1995). 

ENTDLAY; Delayed entry (Continuous) 

DVE944; Dev. English required (Binary, yes=l, no=0) 

DVM944; Dev. math required (Binary, yes=l, no=0) 

MPLUS; Dev. math and another develop, area required (Binary, yes=l, no=0) 
DVR944; Dev. reading required (Binary, yes=l, no=0) 

EFL944; English as a foreign language required (Binary, yes=l, no=0) 

DVTOTL; Number of dev. areas required (Continuous) 

Environmental Variables 

CURCH; Percent, of semesters with curriculum change (Continuous) 

GOALCH; Percent, of semesters with attendance goal change (Continuous) 
REASONCH; Percent, of semesters with attendance reason change (Continuous) 
TERM: Number of semesters attended (Continuous) 

MJ3TERM: Number of first three major semesters attended (Continuous) 

SMTERM: Number of summer semesters attended (Continuous) 

LOADMJ: Average major semester credit hour load (Continuous) 

SITES; Percent, of semesters with combined attendance locations (Continuous) 
SCHS: Percent, of semesters with combined attendance schedules (Continuous) 

CDY 1 : Cumulative first-year credit hours earned (Continuous) 

CDEN; Cumulative credit hours earned (Continuous) 

GPA: Cumulative GPA (Continuous) 

DISPG: Percent, of semesters with good academic standing (Continuous) 
DVCRS944: Number of dev. courses taken in fall 1994 (Continuous) 
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Table 1. (Continued) 

DVCRS: Number of dev. courses taken since fall 1994 (Continuous) 
DVGDPP: Percent, of dev. courses taken with passing grade (Continuous) 
DVCOMP: Number of dev. areas completed (Continuous) 

PAID; Percent, of semesters with financial aid (Continuous) 

Outcome Variable 

OUTCM: Academic outcomes (Binary, achiever=l, nonachiever=0) 



Results 

The likelihood-ratio-based forward stepwise variable selection terminated at step 
6 in the logistic regression analysis when no more variables met entry or removal criteria. 
Table 2 provides descriptive statistics of the five environmental variables in the final 
model (RACE is excluded from the table for its binary nature). 



Table 2. 



Descriptive Statistics of Selected Environmental Variables 

(N=l,249) 



Variable Name 


Mean 


Std. Deviation 


LOADMJ 


8.7 


3.2 


CDEN 


16.8 


19.8 


GPA 


1.6 


1.2 


DISPG 


.6 


.4 


DVCRS 


2.0 


1.8 



The goodness of fit of the regression model was satisfactory. Table 3 indicates 
that this model had an overall 94.96 percent correct prediction rate. Table 4 presents the 
change in -2LL statistic from the initial model containing the constant only before step 1 
to the final model after step 6. As another measure of how well the estimated model fits 
the data, -2LL is the product of -2 times the log of the likelihood with the value of 0 for a 
perfect match. The decrease of its value from 1 124.97 before step 1 to 331.22 after step 6 
should be regarded as a significant improvement. The difference between the two -2LL 
values is listed in Table 4 as the model chi-square, and the step chi-square represents the 
change between the last two steps. For these two chi-squares, their degrees of freedom 
(df) and significance level (Sig) are also presented. 
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Table 3. 

ClassiHcation Table for OUTCOM 


Observed 


Predicted 


Percent 




Yes No 


Correct 


Yes 

No 


168 40 

23 1,018 


80.77% 

97.79% 




Overall 


94.96% 




Table 4. 

Goodness of Fit for the Regression 
Model 


Name 


Value 


df Sig 



-2 LL before step 1 


1124.9 

7 


-2LL after step 6 


331.61 


Goodness of Fit 


992.61 


Model Chi-square 


793.74 


Step Chi-square 


5.78 



Table 5 displays the parameter estimates for the six variables in the final model 
and their related statistics. They include the estimated regression coefficients (B) and 
standard errors (S.E.), the Wald statistic for testing hypotheses about the coefficients 
(Wald) and degrees of freedom (df) and significance level (Sig), the R statistic for 
measuring the partial correlation between the dependent variable and each of the 
independent variables, and the odds of an event occurring (Exp(B)). Of the six 
significant predictors of the 

academic outcomes of the underprepared students, CDEN (R=.35) and GPA (R=.17) 
were more closely related to the dependent variable OUTCM, and DISPG 
(Exp(B)= 19.47) could change the odds of becoming an achiever most dramatically. 
Interestingly, there was a negative coefficient for DVCRS, revealing a negative 
relationship between the number of developmental courses taken and the academic 
outcomes. 
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Table 5. 



Parameter Estimates for the Regression Model 



Variable 


B 


S.E. 


Wald df 


Sig 


R 


Exp(B) 


CDEN 


0.15 


0.01 


135.95 


1 


0.00 


0.35 


1.16 


DISPG 


2.97 


1.11 


7.17 


1 


0.01 


0.07 


19.47 


GPA 


1.60 


0.28 


32.94 


1 


0.00 


0.17 


4.94 


LOADMJ 


0.20 


0.06 


13.01 


1 


0.00 


0.10 


1.23 


DVCRS 


-0.19 


0.08 


5.50 


1 


0.02 


-0.06 


0.83 


RACE 


1.28 


0.42 


9.43 


1 


0.00 


0.08 


3.59 


Constant 


-14.73 


1.60 


84.43 


1 


0.00 







Discussion 

The results from the logistic regression analysis verified some findings of 
previous research that cumulative credit hours earned, good academic standing, 
cumulative GPA, course load, and the number of developmental courses taken have 
varying degrees of impact on the academic progress of underprepared community 
college students (Boughan & Clagett, 1995; Campbell & Blakey, 1996; Long & Amey, 
1993). As most of the variables remained in the final logistic regression model reflected 
the environmental experiences of college students in general, some people may be 
tempted to suggest that this model may not well explain the academic outcomes of 
underprepared conununity college students. This, however, may not be true. First, 
whether the student is prepared or underprepared for college -level course work, the 
outcomes of the individual’s academic endeavors bear some common characteristics that 
can be evaluated by measures such as course load, GPA, and academic standing. Second, 
with increasingly vigorous enforcement of mandatory placement in developmental 
education courses, the mere attainment of some GPA points and credit hours marks an 
underprepared student’s upward movement from developmental courses to college-level 
courses, since developmental courses do not carry credit toward a certificate or a degree 
in general. There is some obvious cohesion among four out of the six predictors in the 
logistic regression model: With an adequate course load in major semesters (LOADMJ), 
an underprepared student is expected to earn more credit hours (CDEN); as the number of 
credit hours grows almost simultaneously with the individual’s academic level, the 
student is more likely to have a higher GPA and to remain in good academic standing 
(DISPG). There is only one academic predictor in the model that directly measures the 
effect of developmental education (DDVCRS). The regression coefficient is negative 
because the number of developmental courses taken since fall 994, whether 
conscientiously or not, shows the severity of the student’s academic deficiencies. An 
interesting finding from this study was that race/ethnicity, the only input variable in the 
logistic regression model, was positively related to academic outcomes. Since white and 
non-white students were coded as 0 and 1 respectively for the data analysis, it might be 



inferred that minority underprepared students fare better in developmental education, and 
subsequent college-level education programs. Although researchers (e.g., Gates & 
Creamer, 1984; Losak, 1983) have found that black conununity college students were 
better persisters than their white peers, this difference in academic outcomes has not been 
substantiated with empirical data. 

It might be argued that Astin’s (1991, 1996) I-E-O model was not fully supported 
by this study. This could be an artifact of the practical constraints that limited the way 
the data were collected for the study. For example, the two variables HSTYPE and B90, 
were both useful input variables. However, since these measures were based on groups 
(i.e., schools and neighborhood) rather than on individual persons, the within-group 
variation of these variables were unavoidably suppressed. Also, our research office relies 
on the Transfer Student System of Maryland Higher Education Commission for transfer 
student information. Yet this system can only identify PGCC students who transferred to 
a Maryland 4-year public post-secondary institution with at least twelve credits about a 
year ago. As a result, some “dropouts” in our student database with high GPAs might be 
real achievers who may have long transferred to a private institution or an out-of-state 
institution. There were other factors that may have affected the results of this study. 
Because of the limited resources, we were not able to conduct a comprehensive model- 
based student survey to collect data. Besides, our selection of only underprepared 
students as the target of our research might methodologically nullify the use of some 
developmental education-related variables in our analysis. 

Conclusion 

A recent logistic regression analysis of the longitudinal data of the underprepared 
students in the fall 1994 PGCC new entrants identified six significant predictors of these 
students’ academic outcomes. As the predictors were largely academic, it seems that the 
underprepared students should be encouraged to work harder, and to persist at the 
college. The current academic achievement rate of PGCC underprepared students is low. 
However, with our devoted faculty, administration, and Educational Development 
Program personnel, the situation is improving. The college will continue to enroll a large 
number of underprepared students, yet it will never become a remedial education 
institution (Bickford, Clagett, James, & Taibi, 1998). 
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NEAIR 25th Anniversary Conference 



Saturday, November 14, 1998 




1:00- 5:00 p.m. 

Nicholas Moore 


Conference Registration 


2:00 - 5:00 p.m. 

Reynolds 


Newcomers to Institutional Research, Part 1 

This workshop is designed for new practitioners 


Karen Bauer 

Assistant Director of Institutional 
Research and Planning 
University of Delaware 
NEAIR President-elect 


who engage in IR activities. Using the AIR 
monograph, Strategies for the Practice of 
Institutional Research, the workshop addresses key 
components of IR including defining critical issues 
for institutional research, identifying sources of 
data, developing factbooks and other reports, and 
conducting effective survey research for assessment 
and evaluation. The main focus is a presentation of 
general concepts and practical strategies for the 
implementation of continued development of 
effective IR at many schools, regardless of size or 
type. 

Workshop 


2:00-5:00 p.m. 

Flower 


Research Design Ideas for Institutional Researchers 


Anne Marie Delaney 
Director of Institutional Research 
Babson College 


The goal of this workshop is to enhance 
institutional researcher’s ability to translate data 
into information and to transform reporting into 
research. Objectives include enabling participants 
to prepare methodologically sound research reports 
for their institutions and research proposals for 
professional conferences. The workshop will 
demonstrate how the institutional researcher can 
use principles of research design and selected 
research techniques to transform data collection 
activities into meaningful research projects. Ideas 
for the workshop will be based on research projects 
completed by the presenter as well as on actual or 
proposed studies of interest to the participants. 




Workshop 
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NEAIR 25th Anniversary Conference 


Saturday, November 14, 1998 cont 


.inued 


2:00 - 5:00 p.m. 

Ballroom El 

Craig Billie 

Associate for Institutional 
Research 

State University of New York, 
System Administration 

Linda LeFauve 

Director of Institutional Research 
Davidson College 


An Introduction to Statistical Sampling Studies 

From opinion surveys to election polls to (possibly) 
the next decennial census, examples of statistical 
sampling studies abound in social science and 
educational research. This workshop will cover the 
basic types of sampling designs, address the 
determination of sample size, and examine the issue 
of bias due to non-response. These topics will be 
developed using specific sampling studies the 
presenters have conducted at their respective 
institutions. 

Workshop 


2:00 - 5:00 p.m. 

Temple University 

Robert Toutkoushian 
Executive Director 
Office of Policy Analysis 
University System of New 
Hampshire 


IR Applications of Regression Techniques 

The workshop will illustrate some of the ways in 
which regression techniques can be applied to 
institutional research. Techniques to be reviewed 
include multiple regression, logit and probit 
analysis and poisson regression. Some familiarity 
with statistics and regression analysis will be 
assumed for participants. The workshop will focus 
on how to apply these techniques. 

Workshop 


2:00 - 5:00 p.m. 

Temple University 

William Lauffer 

Department Chair, Engineering 

Technology 

Prince George’ s Community 
College 


Creating a Web Page Using Netscape 

Learn how to create a home page without writing 
HTML code or using an expensive software 
package. You will learn how to use Netscape’s 
editor (included with the 3.0 Gold and 4.0 versions) 
to display text, images, and backgrounds. You will 
also learn how to create links to other pages at your 
own site and around the World Wide Web. You are 
guaranteed to leave the workshop with a working 
home page. We will provide sample files and 
images for you to work with, but you may wish to 
bring your own (in *.jpg or *.gif format). 

Workshop 
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NEAIR 25th Anniversary Conference 


Saturday, November 14, 1998 coni 


tinned 


6:00 - 7:00 p.m. 
Hamilton 


Newcomers’ Orientation and Reception 


Sunday, November 15, 1998 




8:00 a.m. -4:30 p.m. 
Foyer 


Registration 


9:00 a.m. - noon 

Reynolds 
Karen Bauer 

Assistant Director of Institutional 
Research and Planning 
University of Delaware 
NEAIR President-elect 


Newcomers to Institutional Research, Part 2 
Continuation; Part 1 is a pre-requisite. 

Workshop 


9:00 a.m. - noon 

Flower 


Survey Production Mechanics 

Recently, college administrators, professors and 


Ed Silverman 


government agencies want IR to “do a survey.” 


Director, Institutional Research 


This workshop will show how a one-person office 


City College of New York 


uses available computer technology to create, scan, 
analyze, and report on surveys. 

Workshop 
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NEAIR 25th Anniversary Conference 



Sunday, November 15, 1998 continued 



9:00 a.m. - noon 
Ballroom El 

Michael Middaugh 
Assistant Vice President for 
Institutional Research and 
Planning 

University of Delaware 
1989-90 NEAIR President 



Developing Appropriate Measures of Teaching Workload 

This workshop provides a “nuts and bolts” 
discussion of concrete strategies for building a 
campus data base that lends itself to a thorough 
analysis of faculty teaching loads, instmctional 
costs, and research and service activity. The Boyer 
Commission on Educating Undergraduates in 
Research Universities recently decried the quantity 
and quality of undergraduate contact with tenured 
and tenure track faculty at those institutions, 
suggesting they are not getting full value for their 
tuition dollar. Less harsh, but still real comparable 
criticism has been directed at all of higher 
education, especially with regard to costs. With 
growing external pressures for greater 
accountability from higher education, it is 
imperative that institutions be able to respond to the 
question, “Who is teaching what to whom, and at 
what cost?” This workshop provides the necessary 
tools for responding to that and other questions 
currently being directed at colleges and universities. 



Workshop 



9:00 a.m. - noon 

Temple University 

Heather M. Bames 
Information Technologist II 
Keene State College 



Designing Professional Presentations Using Microsoft 
Power Point 

This workshop is designed for the institutional 
research professional who would like to learn how 
to design effective computerized presentations. 

This introductory session using Power Point for 
Office97 will include everything the participant 
needs to start creating great presentations, including 
animation, graphics, presenter’s notes, handouts, 
and saving as HTML. 



Workshop 



NEAIR 25th Anniversary Conference 


Sunday, November 15, 1998 continued 


9:00 a.m. - noon 
Temple University 

Mary Ann Coughlin 
Professor of Research & Statistics 
Springfield College 


Statistics for Institutional Research 

Basic ideas in statistics will be covered in a way 
useful as an introduction or as a refresher to 
statistics. Descriptive statistics, sampling and 
probability theory as well as the inferential methods 
of chi-square, t-test and Pearson’s r will be covered. 
May be taken with or without the follow-up 
advanced workshop. 




Workshop 


Noon - 1 :00 p.m. 
Ballroom D 


Lunch break 


1 :00 - 4:00 p.m. 

Reynolds 


Office Management and Information Dissemination 
Strategies for New Directors of Institutional Research 


Craig Clagett 

Director of Institutional Research 
and Analysis 

Prince George’s Community 
College 

1997-98 NEAIR President 


Designed for institutional researchers, who have 
recently become directors, this workshop focuses 
on office management strategies and techniques for 
effective information dissemination. To maximize 
IR impact, we need to know our institutions and 
their decision environments, operate efficiently to 
create the time for policy-focused research, and 
communicate timely, targeted intelligence. Topics 
covered include environmental scanning, office 
staffing, staff incentive and recognition programs, 
office project management systems, principles of 
tabular and graphical data presentation, print and 
electronic reporting, and office Websites. 




Workshop 



ERIC 



191 



191 



NEAIR 25th Anniversary Conference 


Sunday, November 15, 1998 continued 


1:00 - 4:00 p.m. 

Flower 

Michael McGuire 
Director of Institutional Research 
Georgetown University 
1993-94 NEAIR President 

Jim Trainer 

Director, The Higher Education 
Data Sharing Consortium 
Franklin and Marshall College 


Making Effective Inter-institutional Comparisons 

In this workshop the presenters will discuss the 
various aspects of making effective inter- 
institutional comparisons. The workshop will cover 
topics ranging from selecting appropriate peer 
groups to accessing comparative data and 
ultimately using these data effectively to inform the 
planning and decision-making process. The value 
of inter-institutional cooperation in sharing data 
also will be discussed. 

Workshop 


1:00 - 4:00 p.m.Ballroom El 


Using SAS to Automate Data Presentation on the Web 
Providing data is one of the most important and 


Lynn Atkinson 


time consuming services provided by IR offices. 


Senior Research Analyst & 


This workshop shows SAS users how to present 


Enrollment Coordinator 


data on the Web. Participants will learn how to use 
SAS to create customized HTML pages that include 


Jack Keil 


stylish features such as backgrounds, graphics, and 


Research Analyst 


hyperlinks. Upon completion of the workshop, 
participants will possess the tools and knowledge to 


Peter Partell 


automate data presentation at their home 


Research Analyst 
Binghamton University 


institutions. Knowledge of HTML is beneficial but 
not required. 

Workshop 
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1 :00 - 4:00 p.m. 


How to Customize Reports using Excel Automation 


Temple University 


This workshop is designed to help participants 


Hui-May Chu 


enhance their Excel skills and consists of examples 


Research Analyst 


of interest to institutional researchers. Techniques 


University of Connecticut 


will be shared that reduce duplication of data 
storage, simplify formatting, and increase efficiency 
and accuracy in production of tables and graphs 
from a set of templates. The workshop will be 
divided into three modules: (1) Introduction of cell 
linkages, formulas, and graphs in Excel; (2) Design 
of templates and the how-to of visual basic macros; 
and (3) Discussion and advice for specific projects 
of participants. Both simple examples and 
complicated setups will be illustrated. The intended 
audience should have working knowledge of 
Microsoft Excel. Participants will have hands-on 
experience of macro writing for their specific 
projects. 




Workshop 


1:00 -4:00 p.m. 


Advanced Statistics for Institutional Research 


Temple University 


This workshop will deal with advanced issues in 


Mary Ann Coughlin 


inferential statistics. Topics such as Analysis of 


Professor of Research and 


Variance, Factor Analysis, Multivariate Regression, 


Statistics 


and Logit/Probit models will be covered and 


Springfield College 


contrasted with other statistical tools and 
techniques. A case study approach will be used 
illustrating applications of these statistical 
techniques in institutional research. Open to those 
who have completed the introductory workshop on 
Sunday morning or who have an equivalent 
background. 




Workshop 
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4:00 - 4:50 p.m. 


Special Interest Groups 


Reynolds 


Admitted Student Questionnaire 


Ellen Kanarek 
Vice President 

Applied Educational Research, 
Inc. 

1995-96 NEAIR President 




Flower 


National Council for Research and Planning - Two-year 
Colleges 


Alan Sturtz 




Director, IR and Planning 
Gateway Community-Technical 
College 




Ballroom El 


Higher Education Data-Sharing Consortium 


Jim Trainer, Director 
HEDS Consortium 




Ballroom E2 


The Delaware Study 


Michael Middaugh 
Assistant Vice President for 
Institutional Research and 
Planning 

University of Delaware 
1989-90 NEAIR President 




William R. Fendley 
Director, Institutional Research 
University of Alabama 
AIR President 
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5:00 - 6:00 p.m. 
Ballroom A & B 


The Learning Paradigm and Institutional Transformation 


William J. Flynn 

Dean, Division of Media, Business 
& Community Services 
Palomar College 


Rarely has an analysis of higher education 
provoked as much discussion and debate as an 
article that appeared in the November/December 
1995 issue of Change magazine. It’s title: From 
Teaching to Learning - A New Paradigm For 




Undergraduate Education. The article outlined the 
changes in mission, vision and structure necessary 
to create a learner-centered college. This session 
reviews the main tenets of the Learning Paradigm, 
and offers insight into the tools and techniques that 
could be used to begin an institutional 
transformation. Implications for budgeting 
decisions, facilities design, strategic planning and 
academic revitalization will be addressed. 




Opening General Session 


6:00 -7:00 p.m. 
Foyer 


Presidents’ Reception 


7:00 - 9:00 p.m. 
Ballroom C, D, & E 


Banquet and Entertainment 
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8:00- 11:00 a.m. 
Foyer 


Registration 


7:30- 8:50 a.m. 
Ballroom B, C, & D 


Continental Breakfast 

(Note: Breakfast Workshares start at 8:00 a.m.) 


8:00- 8:50 a.m. 

Reynolds 


Shaping Internet Courses at Harrisburg Area Community 
College: Assessing Student Interest 


Marianne Guidos 
Educational Data Specialist 
Harrisburg Area Community 
College 


To assess student interest in courses offered over 
the Internet at Harrisburg Area Community 
College, credit, noncredit, and distance education 
students were surveyed. Results indicate 
nontraditional students were most interested in 
Internet delivery and the majority of students had 
both the computer experience and resources 
necessary to enroll. 


Moderator: Craig Clagett 


Workshare 


8:00-8:50 a.m. 

Flower 


Do You Know What Those Numbers Really Mean? 
Understanding Financial Performance Indicators 


Judith A. Jaffe 
Policy Analyst Consultant 
Massachusetts Board of Higher 
Education 


Financial indicators are frequently incorporated into 
the performance measurement systems institutions 
use to determine if they are achieving their goals. 
Institutional researchers must familiarize 
themselves with the different indicators, their 
components, and their correct interpretation. This 
session will explore the meaning and use of the 
financial indicators used by public institutions in 
Massachusetts. 


Moderator: Michael Keller 


Workshare 
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Monday, November 16, 1998 continued 


8:00 - 8:50 a.m. 

Cook 

Sherri Noxel 

Statistical Information Specialist 

Barbara Wharton 
Coordinator, Office of Testing 

Ohio State University 


CIRP Data Representativeness; An Exploration of 
Response Differences Between Students Permitting 
Follow-up Research and Anonymous Respondents 

Typically 53 percent of the freshmen at Ohio State 
University permit identification in the Higher 
Education Research Institute’s Fall Freshman 
Survey. These data are increasingly important in 
longitudinal retention and degree progress research. 
Trackable student responses were analyzed and 
evaluated for differences in comparison to the total 
new student population. 


Moderator: Karl Boughan 


Workshare 


8:00 — 8:50 a.m. 

Bromley 


Outcomes and Effectiveness: The Mission, the Plan, the 
Measures 


Alan J. Sturtz 
Director, IR and Planning 
Gateway Community-Technical 
College 


Assessment of institutional effectiveness must 
become more rational, more public, more 
systematic, and more consistently based upon data 
and other objective measures. Regional accrediting 
agencies are now mandating this. This workshare 
covers the what, why, who, how, and when of IE 
and provides one example of an 
assessment/effectiveness plan. 


Moderator: Kenneth Redd 


Workshare 


8:00-8:50 

Claypoole 


The Survey Doctor is In: Extending your Reach through 
Internal Consultation Services and On-line Guidance 


Kim Dolphin Wheaton 
Coordinator of Institutional 
Research 
Bentley College 


At Bentley College, the demand for institutional 
research services exceeds staff resources. Our 
solution: Extend our reach through targeted 
consultations and a detailed, on-line guide to the 
survey research process. The session will describe 
why this is a good idea, how it works, and how to 
share your expertise through technology. 


Moderator: Catherine Watt 


Workshare 
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9:00 - 9:50 a.m. 
Reynolds 


On-line News vs. Traditional Media: Student Preference 
Regarding the Acquisition of Current Events 


Dawn Geronimo Terkla 
Executive Director, Institutional 
Research 

1992-93 NEAIR President 


In the summer of 1 997 a contract was signed with 
Prentiss Hall to obtain university-wide access to 
College Newslink, a web-based newsclipping 
service with stories classified by academic 
discipline. As part of the license agreement, it was 


Jennifer McKnight 
Research Analyst 


stipulated that the university would evaluate the 
product in terms of ease of access, reliability, 
desirability and usefulness. Simultaneously, there 


Tufts University 


was interest on campus in determining how students 
prefer to receive their news information. In an era 
of limited resources, the question was being raised 
whether the university should use funds to provide 
multiple subscriptions to the New York Times or a 
daily electronic news service. This paper will 
describe how the institutional research office went 
about developing a strategy to evaluate College 
Newslink, assess students’ preferences regarding 
the acquisition of current events, and the difficulties 
that were encountered along the way. 


Moderator: Sarah Parrott 


Paper 


9;00 - 9:50 a.m. 

Flower 


The Impact of Outsourcing Auxiliary Enterprises on 
Strategic Indicators Involving Educational Costs 


Joan McDonald 

Assistant Director of Institutional 
Research 


Research results will be presented from a study of 
factors contributing to educational costs in 100 
private institutions. It was conducted to determine 
the extent to which strategic indicators can be 


Barbara S ado w ski 
Assistant to the President for 
Planning and Research 


affected merely by an institution’s decision to 
outsource auxiliary enterprises. The relationship 
between outsourcing, allocation of indirect costs, 
and changes in total educational costs were 


Marywood University 


investigated to determine the variables’ relative 
contributions in predicting changes in total 
educational costs. 


Moderator: Tara Maher 


Paper 
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9:00 - 9:50 a.m. 
Cook 


The Learning Paradigm: Getting Started on Your 
Campus 


William J. Flynn 

Dean, Division of Media, Business 
and Community Services 
Palomar College 


This session will look at ways to start a campus 
dialogue on how colleges and universities must 
change to meet the new challenges of calls for 
accountability, the explosion of technology and 
increased competition. It will begin with a showing 
of the Palomar College award-winning video on the 
Learning Paradigm. The audience will be 
encouraged to discuss ways to involve campus 
constituencies in openly addressing issues of 
change and transformation. 


Moderator: Bill Fendley 


Workshare 


9:00-9:50 a.m. 
Bromley 


Assessing the Effects of Basic Skills Policy Changes at a 
New Jersey Community College 


Carolyn Hoffman 
Research Associate, Institutional 
Research and Planning 
County College of Morris 


With the demise of the New Jersey Department of 
Higher Education in 1994, public colleges and 
universities were granted autonomy in establishing 
basic skills placement procedures and curricula. 
This study utilizes data from the 1993 through 1997 
cohorts to assess the effectiveness of a series of 
policy changes instituted by one New Jersey 
Community College. 


Moderator: Jim Trainer 


Paper 
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9:00 - 9:50 a.m. 


Accreditation Standards and Institutional Effectiveness 


Claypoole 


An increasingly important issue for both regional 


Alan Sturtz 


accrediting agencies and institutions of higher 


Director of Institutional Research 


education is institutional effectiveness: assessing 


and Planning 


how effectively institutional resources are utilized 


Gateway Community-Technical 


and services delivered and how this measurement 


College 


results in institutional improvement. 
Representatives from the NEASC and Middle 
States will discuss how they are implementing this 


Amy Kirle Lezberg 


concept in their standards, implications for colleges 


Associate Director 


and universities and the possibilities for comparable 


Commission on Institutions of 


standards across the regions. An open question and 


Higher Education 

New England Association of 

Schools and Colleges 


answer period will follow. 


Jean Avnet Morse 
Executive Director 




Commission on Higher Education 
Middle States Association of 




Colleges and Schools 


Panel 


10:00 - 10:50 a.m. 


Comparative Financial Aid Measures for Consumer 


Reynolds 


Audiences: What Prospective Students and Parents 
Really Need to Know 


Michael McGuire 

Director of Institutional Research 


Consumer demand for information on college costs 


1993-94 NEAIR President 


has led college guidebooks to focus on financial aid 
in well-intentioned but imprecise ways. This paper 


Ronald Allan 


critiques current financial aid data collection and 


Manager, Data and Research 


reporting methods, and proposes an alternative 


Services Office of Student 


approach that appears to provide more useful 


Financial Services 


information to prospective students and their 


Georgetown University 


parents. 


Moderator: Richard Heck 


Paper 
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10:00- 10:50 a.m. 

Flower 


IR’s Role in the Development of a Technology Strategic 
Plan 


Gayle Fink 

Coordinator of Institutional 
Research 

Michelle Appel 
Research Analyst 

Anne Arundel Community College 


Community college researchers will discuss their 
impact in creating a new Technology Strategic 
Plan. Information from a variety of surveys and 
data sources is being used to develop a 
comprehensive assessment of the college’s 
technology needs and strengths. Additionally, IR's 
service on the planning workgroup will be 
discussed. 


Moderator: Barbara Sadowski 


Workshare 


10:00- 10:50 a.m. 

Cook 


Using Predictive Models to Target Recruitment Initiatives: 
Theory and Practice 


Emily Thomas 
Director of Planning and 
Institutional Research 

Gayle Reznick 
Research Assistant 

SUNY Stony Brook 


College admissions offices are experimenting with 
predictive modeling to target recraitment efforts, 
but a typical use of these models is theoretically 
questionable and may therefore be operationally 
inefficient. The results of an experimental program 
suggest that a different use of predictive modeling 
can be a fruitful collaboration between institutional 
research and admissions staff. 


Moderator: 

Robert Toutkoushian 


Paper 
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10:00- 10:50 a.m. 

Bromley 


Tracking Community College Students to Bachelor’s 
Degree Completion: Methodological Issues in 
Performance Funding 


Keith J. Guerin 

Director, Institutional Research 

and Planning 

County College of Morris 


Performance funding has come to New Jersey, and 
bachelor’s degree completion rates are a funding 
measure. Surprisingly, both community colleges 
and senior institutions are benchmarked for the 
same rates for their transfers. This paper examines 
empirical data and methodological issues associated 
with the validity of this performance indicator and 
its relationship to policy intent. 


Moderator: Paul Carmichael 


Paper 


10:00 - 10:50 p.m. 

Claypoole 


Quasi-experimental Design and the Pre-testing Effect 
Although a conscientious effort to develop sound 


Cherry Danielson 
Research Associate, Research 
Office for Student Affairs 
University of Michigan 


research designs must be a norm within institutional 
research, even the best-laid plans can result in the 
emergence of a pre-testing effect. This paper looks 
at an example of such a study and offers some 
practical ways to deal with the results. 


Moderator; Jason Casey 


Paper 


11:00- 11:50 a.m. 

Reynolds 


Designing Graduate Admission Studies to Influence 
Campus Transformation 


Anne Marie Delaney 
Director of Institutional Research 
Babson College 


This paper presents the research design, significant 
results and strategic policy recommendations from 
a graduate admission study which examined the 
enrollment decision processes of students accepted 
to a newly redesigned Master of Business 
Administration (MBA) Program. The paper 
illustrates the role of institutional research in 
campus transformation by demonstrating how the 
study was used to evaluate the impact of curricular 
reform and the effectiveness of admission strategies 
in recruiting students to choose an innovative 
graduate degree program. 


Moderator: Penny Blackwood 


Paper 
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11:00- 11:50 a.m. 
Flower 


Extracting Data from Our New Administrative Computer 
Systems 


Ray Perry 

Computer Services Manager 
Cecil Community College 

Dan McConochie 
Director of Planning and 
Evaluation 

Howard Community College 


In the past three years, many colleges have 
purchased new administrative computer systems, 
typically SCT Banner or Datatel. These systems 
carry the promise of improved data retrieval and 
analysis tools over the legacy systems that they 
have replaced. This workshare will discuss 
strategies to take advantage of these new 
capabilities on a cooperative basis by sharing 
training and avoiding duplication of report 
programming. 


Moderator: Paula Pelletier 


Workshare 


11:00- 11:50 a.m. 
Cook 


How Much Do Faculty Really Teach? Lessons Learned 


Michael F. Middaugh 
Assistant Vice President for 
Institutional Research & Planning 
University of Delaware 
1989-90 NEAIR President 


This paper revisits an analysis done in 1996, using 
data from the Delaware Study of Instructional Costs 
and Productivity. That analysis found that tenured 
and tenure track faculty generate a much larger 
proportion of undergraduate teaching activity than 
might be expected in light of public criticism of 
American higher education. The 1996 
methodology is replicated using data from the two 
most recent iterations of the Delaware Study. The 
result is a three-year trend line that more clearly 
indicates how much faculty really teach, and 
associated cost and productivity measures. 


Moderator: Marian Pagano 


Paper 
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11:00- 11:50 a.m. 

Bromley 


Math Achievement and Math Course Taking 
This research examines the math achievement 


Denise Nadasen 

Coordinator, Institutional Research 
University System of Maryland 


scores and math course taking patterns of 16,000 
12'*' graders selected form the National Educational 
Longitudinal Study of 1998. Using math 
achievement as the criterion variable and math 
course work as the predictor variables, the amount 
of explained variances attributable to specific math 
courses are identified. Results indicate that 
geometry contributes the most explained variances 
toward math achievement. 


Moderator: Indira Govindan 


Paper 


11:00- 11:50 a.m. 

Claypoole 


The National Student Loan Clearinghouse, TransferTrack 
and the Future 


Kenneth R. Ostberg 
Director, Business Development 
National Student Loan 
Clearinghouse 


The session will discuss our experience with the 
first year of tracking transfers for Student Right-To- 
Know and IPEDS reporting purposes and 
Clearinghouse plans for the future for enriching the 
galaxy of research data that can be furnished to 
institutions and systems. 


Moderator: Ellen Kanarek 


Workshare 


Noon - 1:30 p.m. 
Ballroom B, C, <Sc D 


Luncheon and Business Meeting 


1:30 - 2:00 p.m. 
Foyer 


Networking and Vendor Exhibits 



NEAIR 25th Anniversary Conference 


Monday, November 16, 1998 continued 


2:00 - 2:50 p.m. 
Reynolds 


Description of the UDAES Project: A Study of 
Undergraduate Academic Experiences 


Karen W. Bauer 
Assistant Director, Institutional 
Research and Planning 
University of Delaware 
NEAIR President-elect 


Many IR officials are involved in assessing the 
academic transformations that occur on campus. 
This presentation describes the research design and 
findings from the first of a four-year research 
project, the University of Delaware Academic 
Experiences Study (UDAES). Data gathered from 
265 students in May 1997 will serve as baseline 
measures for growth and change over the next three 
years. 


Moderator: Becky Brodigan 


Workshare 


2:00 - 2:50 p.m. 

Flower 

Wayne Obetz 
Research Associate 

Jane Grosset 

Director of Institutional Research 

Dana Cantu 
Research Assistant 


F rom Database to Data Warehouse 

This workshare will trace the development of a data 
warehouse at the Community College of 
Philadelphia from its initial conception as a 
database used to provide information to the 
program audit process to its present form, a tool 
capable of providing answers to both routine and 
unanticipated queries. 


Bette Irwin 
Research Assistant 




Community College of 
Philadelphia 




Moderator: Robert Ploutz-Snyder 


Workshare 
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2:00-2:50 p.m. 
Cook 


Comparing Administrative Satisfaction in Public and 
Private Higher Education 


J. Fredericks Volkwein 
Director of Institutional Research 


Do administrators in public higher education 
experience different levels of job satisfaction than 
their counterparts in the private sector? This study 


Kelli Parmley 
Doctoral Student 


draws upon a comprehensive array of national data 
on university characteristics, state characteristics 


University at Albany 


and administrative satisfaction. Previous analyses 
have explored the relationship of state regulation to 
administrative satisfaction in public higher 


Shaukat Malik 

Assistant for Institutional Research 
SUNY Stoney Brook 


education. However, there are few studies in the 
higher education management or public 
administration literature that have explored the 
differences in job satisfaction between the public 
and private sectors. Such research is important 
because of the connection in the literature between 
levels of satisfaction and employee productivity and 
managerial turnover. 


Moderator: Keith Guerin 


Paper 


2:00-2:50 p.m. 
Bromley 


Transforming Your Campus: Mixed Methodology in 
Institutional Research 


Tracy Polinsky 
Acting Assistant Director, 
Institutional Research and 
Planning 


Researchers play crucial roles in campus 
transformation via information. This information 
must best represent their institution’s reality. By 
combining quantitative and qualitative data 
collection methods, researchers can help isolate 


Robert J. Brodnick 
Director, Institutional Research 
and Planning 


leverage points for change. This paper reviews 
multiple methods, discusses data integration, and 
gives an example where mixed methodological 
research induced successful transformation. 


Shippensburg University 




Moderator: Mark Eckstein 


Paper 
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2:00 - 3:30 p.m. 

Claypoole 



The Association for Institutional Research Code of Ethics i 
A Message FOR and FROM the Membership 



William R. Fendley 
Director, Institutional Research 
The University of Alabama 
1998-99 AIR President 

Timothy R. Sanford 
Director, Institutional Research 
University of North Carolina- 
Chapel Hill 
1996-97 AIR President 

Dawn Geronimo Terkla 
Executive Director, Institutional 
Research & Planning, Tufts 
University 

1992-93 NEAIR President 
3:30-5:00 p.m. 



Adopted by the Association for Institutional 
Research (AIR) membership in 1992, the Code of 
Ethics represents a statement by the AIR for 
members concerning their competence, their 
professional practice, their responsibility to 
confidentiality, their relationships to the educational 
community and public, and their relationships to the 
craft and the profession. The intent of the AIR 
Code of Ethics was and is to establish guidelines of 
professional conduct and behavior which would 
keep the AIR member in good standing with his/her 
professional colleagues and give credibility to 
his/her work as a professional. 



Panel 

IPEDS Update: Looking to the Future 



Claypoole 



Susan Broyles 
IPEDS Project Director 

Roslyn Korb 
Director, Postsecondary 
Cooperative Program 

National Center for Education 
Statistics 



Any change to the Integrated Postsecondary 
Education Data System (IPEDS) conducted by the 
National Center for Education Statistics (NCES) 
has the potential to impact the work of institutional 
researchers in all types of institutions. Because of 
that potential, it is important to stay informed about 
the direction that IPEDS will be taking in the 
future. This panel will focus on several areas of 
interest to institutional researchers including 
changes to IPEDS for the 1998 - 2000 survey years 
and new methods for reporting these data 
electronically. Panelists will also discuss NCES’ 
plans to redesign IPEDS including progress made in 
the implementation of the new racial/ethnic 
categories and the impact of new (or pending) 
legislation on IPEDS data collection activities. 



Panel 
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3:00 - 3:50 p.m. 

Reynolds 

David X. Cheng 

Director of Institutional Research 
Sheying Chen 

Department of Psychology and 
Sociology 

College of Staten Island, City 
University of New York 


Factors Affecting Grading Practice 

This study addressed a practical concern of grade 
inflation and a theoretical interest in the potential 
factors affecting grading practice. Empirical data 
from the student information system at the College 
of Staten Island, City University of New York were 
utilized. Implications for academic policy making 
and effective intervention are discussed, and 
important measurement and analysis issues in 
grading research are also indicated. 


Moderator: Reneva Sweet 


Paper 


3:00 - 3:50 p.m. 
Flower 


The Relationship between Student Success in College and 
Assessment for Remedial Assistance 


Michael J. Keller 

Director of Policy Analysis and 

Research 

Monica E. Randall 

Policy Analysis and Research 

Specialist 


This paper examines the success rates of students at 
Maryland community colleges and public four-year 
campuses based on the amount of remedial 
assistance for which they were assessed in math, 
English and reading: none, one area, two areas, or 
all three areas. Breakdowns by gender and race are 
included. 


Maryland Higher Education 
Commission 




Moderator: Judith Jaffe 


Paper 
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3:00 - 3:50 p.m. 
Cook 


Images of Colleges and Universities in Popular Cinema 


Marian Pagano 
Associate Provost 
Columbia University 

1994-95 NEAIR President 


The impact of the image of colleges and 
universities projected to the public in popular 
cinema is described and discussed while viewing 
clips from several influential and/or popular films 
that had a college or university as a central focus. 
The impact that such images have on the public’s 
opinion of higher education will be discussed along 
with data that might support the power of these 
images. 


Moderator: Jennifer McKnight 


Paper 


3:00 - 3:50 p.m. 

Bromley 


The Effects of Institution-funded Student Financial Aid on 
College Finances, Student Enrollments, & State Financial 
Aid Policies, FY90-96 


Kenneth E. Redd 
Senior Research Associate 
Sallie Mae, Inc. 


Since 1990, institutional financial aid provided by 
public four-year colleges and universities has 
jumped by 62 percent, while aid awarded by private 


Travis Reindl 
Policy Analyst 

American Association of State 
Colleges and Universities 


colleges rose by 61 percent. Much of this increase 
occurred at highly selective institutions. This study 
uses IPEDS data to examine trends in institutional 
aid at public and private institutions, compares 
these trends to changes in other education-related 
expenditures, and uses NPSAS and FISAP data to 
examine changes in the income levels of students 
who attend these colleges. The study also looks at 
how changes in institutional aid may have been 
influenced by changes in state financial aid policies. 


Moderator: Marilyn Blaustein 


Paper 
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4:00-4:50 p.m. 


Using Microsoft Access to Facilitate the Sampling and 


Reynolds 


Distribution of an Alumni Survey 


Karen DeMonte 


Microsoft Access was used in an alumni survey that 


Research Analyst 


required a specific matched sample of respondents. 
The presenters will discuss how Access was used to 


Karen Bauer 


select the respondents, organize the dataset, and 


Assistant Director, Institutional 
Research and Planning 
NEAIR President-elect 


create labels for the initial and follow-up mailings. 


University of Delaware 




Moderator: Tracy Hunt- White 


Workshare 


4:00-4:50 a.m. 


Why Students Send Their Test Scores to a College: A 


Flower 


Case Study 


Robert K. Toutkoushian 


In this study, I show how data from the College 


Executive Director, Office of 


Board can be used to examine whether a student 


Policy Analysis 


decides to have his or her SAT scores sent to a 


University System of New 


particular college. The study relies on both time- 


Hampshire 


series and cross-section data, and special emphasis 
is given to the importance of findings for 
institutional research. 


Moderator: Dawn Terkla 


Paper 


4:00 - 4:50 p.m. 


Evaluating Faculty Work: Merit Pay and Post-tenure 


Cook 


Review at a Liberal Arts College 


Catherine E. Watt 


Effective and reliable evaluation of faculty work at 


Director of Institutional Research 


a liberal arts college is a complicated process that 
attempts to quantify numerous activities. Research 


Jennifer L. Vest 


endeavors are the easiest to measure, but teaching 


Assistant to the Dean for Retention 


and service are also important. This workshare 


Research 


presents an evaluation schema that could be used 
across disciplines and throughout a professor’s 


Washington College 


career. 


Moderator: Mike Middaugh 


Workshare 



o 
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4:00 - 4:50 p.m. 

Bromley 


An Environmental Scanning Formula for Community 
Colleges 


Marcia M. Lee 

Director, Institutional Research 
and Planning 

Westchester Community College 


Most community colleges have a mission 
substantially more focused than four-year 
institutions, specifically to serve the local 
community in providing postsecondary education 
and workforce training. To do this, a periodic 
environmental scan is essential, yet many 
institutional research offices shy away from 
incorporating this into their routines. This paper 
proposes a five-part formula for environmental 
scans, including population trends, economic 
trends, job market growth trends, high school 
graduation trends, and commutation trends, 
designed to make the project easier, focused, and 
usable the next time around. 


Moderator: Jennifer Brown 


Paper 


5:00 - 6:00 p.m. 

Ballroom B, C, &D 


Providing the Technology Infrastructure to Support 
Innovation in Teaching and Learning 


David Hollowell 
Executive Vice President 
University of Delaware 


General Session 



o 

ERIC 
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NEAIR 25th Anniversary Conference 


Monday, November 16, 1998 continued 


6:00 -6:50 p.m. 


Special Interest Groups 


Reynolds 




Jason P. Casey 
Director, Research Center 


Association of Independent Colleges and Universities of 
Pennsylvania 


Penny Blackwood 
Research Associate 




Association of Independent 
Colleges and Universities of 
Pennsylvania 




Flower 


SUNY AIRPO 


Jane Hammond 

Tompkins-Cortland Comm. Coll. 




Cook 


New Jersey Association for Institutional Research 


Eleanor Swanson 
Director, IR 
Monmouth University 




Bromley 


Connecticut Association for Institutional Research 


Corby Coperthwaite 
Director, IR 

Manchester Comm-Tech College 




Claypoole 


Catholic Colleges and Universities 


Linda Winkler 

Director of IR and Planning 

Mount Saint Mary’s College 





o 
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NEAIR 25th Anniversary Conference 



Tuesday, November 17, 1998 




7:30 -8:50 a.m. 
Ballroom B, C, & D 


Continental Breakfast and Table Topics 


Emily Thomas 
Paula Pelletier 


PeopleSoft and Institutional Research 
A number of schools are implementing PeopleSoft 


SUNY Stony Brook 


systems to manage student, financial and personnel 
data. This table session will give their IR 
professionals the opportunity to meet with each 
other, share information, and discuss issues on 
which they are working or about which they are 
concerned. 


Peggye Cohen 

George Washington University 


Banner and Institutional Research 

This session is offered to meet the needs of 
institutional researchers who use Banner. We will 
discuss topics such as reporting from Banner, 
implementing and upgrading Banner, the upcoming 
SCT Summit conference, etc. Hopefully, we can 
share experiences — both positive and negative — 
and learn from one another. Both new and 
experienced Banner users are welcome. 


Sarah Parrott 

Dawn Geronimo Terkla 

Jennifer McKnight 


Using New Scanning Technology with Principia Remark 
Software 

Aimed at IR professionals who currently use 


Tufts University 


Remark Office OMR, or those who are searching 
for a scanning system. This discussion focuses on 
how our office outgrew our desktop scanner and 
our trials and tribulations in choosing a new, faster 
document scanner that would work with Remark 
Office OMR. 



NEAIR 25th Anniversary Conference 


Tuesday, November 17, 1998 continued 


7:30 — 8:50 a.m. 
Ballroom B, C, <ScD 


Continental Breakfast and Table Topics continued 


Donald R. Jansiewicz 
Carroll Community College 


In the Trenches: IR and Learning Outcomes 

This table topic outlines a clinical approach that 
institutional researchers can use in developing and 
measuring learning outcomes. The political 
economy of academe is examined along with 
obstacles to faculty introspection. The presenter 
suggests how researchers can leverage their 
expertise to create a “culture of accountability” and 
identifies successful/unsuccessful strategies. 


Jane Grosset 
Community College of 
Philadelphia 


A Description of the Process Used to Develop Performance 
Indicators for Assessing Institutional Progress in 
Achieving Technology-Related Goals 

The topic for this table discussion is a description of 
the process that was used to develop indicators for 
use in evaluating institutional progress in achieving 
the goal of ensuring student and institutional 
success through the expansion of technology usage. 
This process included the following steps; 
conceptualization, measurement, data collection, 
and assessment. 
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Tuesday, November 17, 1998 continued 


7:30-8:50 a.m. 


Continental Breakfast and Table Topics continued 


Ballroom B, Q <Sl D 




Dana A. Cantu 


Aligning Institutional Research with Campus 


Community College of 


T ransformation 


Philadelphia 


Offices of institutional research have become 
increasingly responsible for providing much of the 
information necessitated by campus transformation 
efforts occurring nationwide. Phrases like 
“Strategic Planning” and “Total Quality 
Management” are heard from campus to campus. 
Yet, what are campus transformation efforts 
designed to achieve, and what have they achieved 
thus far on our campuses? How can Institutional 
Research offices cope with the demands of future 
change? The aim of this table topic is to expand 
participants’ understanding of campus 
transformation processes as well as provide them 
with a venue for the discussion of positive and 
negative changes, which have resulted from the 
process. 


Reneva Sweet 


Racial Coding 


Edinboro University of 




Pennsylvania 


A current, efficient, and accurate means of data 
collection and dissemination is inherent in our data 
integrity initiative. Edinboro University of 
Pennsylvania recognized the urgent need for 
implementation of a race/ethnicity policy. Our 
policy remains within the guidelines of the 0MB 
directives, yet is tailored to fit Edinboro’ s unique 
needs. It is our intention to share this policy, along 
with all processes and procedures, in order to aid 
other’s adaptation to the new federal standards. 
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NEAIR 25th Anniversary Conference 


Tuesday, November 17, 1998 continued 


7:30-8:50 a.m. 
Ballroom B. C, & D 


Continental Breakfast and Table Topics continued 


Bob Yanckello 

Central Connecticut State 

University 


Focus Group 1: The Newport Conference 

A focus group to elicit information on the strengths 
and weaknesses of this year’s conference led by the 
program chair of next year’s conference. By 
invitation. 


Nancy Reiser 

University of Rhode Island 


Focus Group 2: The Newport Conference 

A focus group to elicit information on the strengths 
and weaknesses of this year’s conference led by the 
local arrangements chair of next year’s conference. 
By invitation. 


8:00 - 8:50 a.m. 


Breakfast Workshares: In break-out rooms indicated 


8:00-8:50 a.m. 

Reynolds 

Joseph E. Revelt 

Director of Institutional Research 
Millersville University of 
Pennsylvania 


Report Writers - A Technical Solution to Data Needs 

Report writers - software packages used to query a 
database - have evolved significantly in recent 
years. Modem report writers can be installed on 
PC’s, networks, etc. and provide numerous output 
options including spreadsheets, web pages, etc. 
This presentation will briefly demonstrate such a 
product and illustrate how Millersville University 
selected one. 


Moderator: Steve Thorpe 


Workshare 
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Tuesday, November 17, 1998 continued 


8:00 - 8:50 a.m. 


Breakfast Workshares (continued): In break-out rooms 
indicated 


8:00 - 8:50 a.m. 

Flower 


Designing a Faculty Workload Data Warehouse Data 
Mart 


Andrew S. LaManque 
Research Analyst, Office of the 
Vice President for Academic 
Affairs 

Eric Spear 

Data Systems Manager 
Office of Institutional Studies 

University of Maryland, College 
Park 


This workshare will present information on the 
migration of faculty workload data from SAS data 
files to an oracle based data mart in the University 
of Maryland Data Warehouse. The presentation 
will cover both the technical aspects of data 
warehousing and BrioQuery” dynamic querying 
technology, as well as lessons learned in 
transforming a workload system designed to 
produce a state report, into a useful tool for 
university administrators. 


Moderator: Ray Perry 


Workshare 


8 : 00 - 8:50 a.m. 
Cook 


Establishing a National Survey Scheme for Researching 
Distance Education 


Yuko Mulugetta 

Director of Research and Planning 
Analysis for Admissions and 
Financial Aid 
Cornell University 


There is a growing awareness of the need to survey 
various aspects of distance education. Thus, a 
group of representatives of several national 
organizations have started organizing such efforts. 
A progress report on this endeavor will be 
presented along with available statistics and 
academic research on distance education, as well as 
major players and new technologies identified. 


Moderator: Dan McConochie 


Workshare 



NEAIR 25th Anniversary Conference 


Tuesday, November 17, 1998 continued 


8:00 - 8:50 a.m. 


Breakfast Workshares (continued): In break-out rooms 
indicated 


8:00 - 8:50 p.m. 

Bromley 


What Do Prospective Graduate Students Want? Using 
Focus Groups to Find Out 


Michael Duggan 

Director of Enrollment Research & 
Planning 

Tara Mahar 

Assistant Director of Enrollment 
Research and Planning 


In this session we will discuss the planning and 
implementation of a series of blind focus groups 
exploring what prospective graduate students look 
for in program advertisements. The discussion will 
also include the results of a survey that participants 
completed regarding how they make decisions on 
which program to pursue. 


Jennifer Hanley Ross 
Associate Director 
Creative Services/ Advertising 




Suffolk University 




Moderator: Helen Schneider 


Workshare 


8:00 - 8:50 a.m. 

Claypoole 


Using Graphics for Single- and Multi- Institutional 
Analyses in Institutional Research 


Jason P. Casey 
Director of Research 
Penny Blackwood 
Research Associate 

Association of Independent 
Colleges and Universities of 
Pennsylvania 


Thorough treatment of data typically involves 
various stages of exploration, diagnosis, analysis, 
and presentation. Each of these activities is aided 
by the use of graphical tools. The presenters will 
demonstrate a number of graphical techniques in 
addressing problems in these areas. Graphs created 
in several packages will be demonstrated. 


Michael D. McGuire 
Director of Institutional Research 
Georgetown University 
1993-94 NEAIR President 




Moderator: Linda Winkler 


Workshare 
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Tuesday, November 17, 1998 continued 


9:00-9:50 a.m. 

Reynolds 


Financing Education, An Analysis From the Results of a 
Survey of Graduates 


Qing Lin Mack 

Associate in Institutional Research 

Jennifer Brown 
Executive Officer 
Academic Affairs and Research 
Connecticut State University 
System 

1996-97 NEAIR President 


At the four universities of Connecticut State 
University, we graduate about 4,000 students a year 
with Bachelor’s degrees. Since 1989, we have 
collected data on how our students finance their 
education at CSU. In our presentation, we will 
examine trends in student patterns of educational 
financing against a background of economic 
downturn and changes in financial aid policies. 


Connecticut State University 




Moderator: Patty Murphy 


Paper 


9:00-9:50 a.m. 

Flower 


Using Cluster Analysis for Transcript Analysis of Course- 
taking Patterns 


Wayne Obetz 

Research Associate 

Office of Institutional Research 

Community College of 

Philadelphia 


In order to improve the delivery of services to 
students, colleges must examine the curricular 
experiences of those students. Typically this would 
call for a transcript analysis, a costly and time- 
consuming process. This study makes use of 
cluster analysis to identify discrete groups of 
A.G.S. graduates form a community college. 


Moderator: Cherry Danielson 


Paper 
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Tuesday, November 17, 1998 continued 



9:00 - 9:50 a.m. 
Cook 


The National Postsecondary Education Cooperative: Up 
Close and Personal 


Phyllis A. Fitzpatrick 
Director of Management 
Information 
Fairfield University 


An opportunity to inform NEAIR members of the 
National Postsecondary Education Cooperative 
(NPEC): its origins, its progress since inception in 
1994, its mission and its membership make-up. 


Dawn Geronimo Terkla 
Director of Institutional Research 
Tufts University 
1992-93 NEAIR President 




Moderator: Denise Krallman 


Workshare 


9:00 - 9:50 a.m. 
Bromley 


The Assumptions Underlying Concepts of Performance 
Budgeting 


Toby Milton 

Director of Institutional Research 
Essex Community College 


This presentation examines the concept of 
"performance 

budgeting", places it in the context of other budget 
concepts, 

describes types of performance budgeting, analyzes 
assumptions 

underlying performance budgeting, and notes 
relationships between 

performance budgeting and earlier budget reform 
concepts. Based on 

these examinations, it then questions the feasibility 
and desirability 

of governments and institutions implementing 

performance budgeting 

practices. 


Moderator: Craig Clagett 


Paper 



NEAIR 25th Anniversary Conference 



Tuesday, November 17, 1998 continued 



9:00 - 9:50 a.m. 



New Approaches to the Analysis of Academic Outcomes 



Claypoole 



Karl Boughan 
Supervisor of Institutional 
Research 

Prince George’s Community 
College 

1997 Best Paper 



Using two advanced structural modeling 
methodologies - path analysis and cluster analysis - 
parallel models of the academic programs at Prince 
George’s Community College were developed, 
supported by data tracking the Fall 1990 entering 
cohort over a period of six years. Path analysis 
revealed the centrality of student attitude factors 
(motivation, flexibility, academic gamesmanship) 
to study career success compared with the lesser 
impacts of social background, college preparedness, 
and various process variables. Cluster analysis 
identified several varieties of success-prone 
students, as well as three different student sub- 
bodies, each highly problematic for distinctive 
reasons. 



Moderator: Anne Marie Delaney 



Paper 



NEAIR 25th Anniversary Conference 


Tuesday, November 17, 1998 continued 


10:00- 10:50 a.m. 

Reynolds 


Weaving the Fabric of a Successful Transition: 
Collaborative Research Efforts between Community 
Colleges and Public Schools, a Maryland Model 


Michelle Appel 

Anne Arundel Community College 
Karl Boughan 

Prince George’s Community 
College 

Gayle Fink 

Anne Arundel Community College 


Institutional research personnel and policy analysts 
will discuss the history of Maryland’s Student 
Outcomes Assessment Report (SOAR), factors 
leading to collaborative community college-public 
school research, highlights of collaborative projects, 
and common problems, successes and findings from 
these projects. The impact of these projects on state 
and institutional policies will be highlighted. 


Ruth Garies 
Montgomery College 




Yun Kim 

Charles Community College 




Dan McConochie 
Howard Community College 




Koosappa Rajasekhara 
Dundalk Community College 




Monica Randall 
Maryland Higher Education 
Commission 


Panel 
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Tuesday, November 17, 1998 continued 


10:00 - 10:50 a.m. 

Flower 

Sarah Parrott 
Research Analyst 

Jennifer McKnight 
Research Analyst 

Tufts University 


They’ll Surf but they Won’t Swim: Student Reluctance to 
Apply to College Online and Implications for Web-based 
Survey Research 

Why are college -bound students happy to surf the 
Internet for college information but reluctant to 
apply to college online? Using a large, multi- 
institutional sample of entering freshmen, this study 
examines the reasons they give for not applying 
online, and uses the information to develop 
strategies for institutional researchers who plan to 
conduct Internet-based surveys. 


Moderator: Eleanor Swanson 


Paper 


10:00 - 10:50 a.m. 

Cook 


Trends in the Admitted Student Questionnaire and 
Admitted Student Questionnaire Plus 


Ellen Kanarek 
Vice President 

Applied Educational Research, 
Inc. 

1995-96 NEAIR President 


This workshare will describe trends in the College 
Board’s Admitted Student Questionnaire and the 
Admitted Student Questionnaire Plus since the 
service was first offered in 1988. In addition to 
comparisons between early and recent participants 
as a group, the presentation will examine selected 
colleges that used the survey regularly during the 
period. 


Moderator: Karen Bauer 


Workshare 


10:00 - 10:50 a.m. 

Bromley 
Cheryl Beil 

Director, Academic Planning and 
Assessment 

The George Washington 
University 


Up to their Nose-rings in Credit Card Debt 

How student-loan debt, student over-employment, 
and credit card abuse interact and impact the 
undergraduate experience and contribute to student 
attrition will be the focus of this presentation. 
Findings from a survey of 1,000 undergraduates 
exploring these issues will be presented and 
discussed. 


Moderator: Andrew LaManque 


Paper 
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Tuesday, November 17, 1998 continued 



10:00-10:50 a.m. 

Claypoole 



Assessing Academic Outcomes of Under-prepared 
Community College Students: A Structural Equation 
Modeling Approach 



Jishen Zhao 

Research and Planning Analyst 
Prince George’s Community 
College 



Moderator: Craig Clagett 



This ongoing study was designed to assess the four- 
year academic outcomes of under-prepared 
students at Prince George’s Community College. It 
updates the profile of these students, and identifies 
the factors affecting their academic progress by 
adopting a structural equation modeling approach. 



11:00 a.m. - noon 
Ballroom B, C, Sl D 



Paper 

Closing General Session: NEAIR Presidents’ Panel 



Robert Lay 

1982-83 NEAIR President 



The Role of Institutional Research in Campus 
Transformation 



Marian Pagano 
1994-95 NEAIR President 

Patrick Terenzini 
1980-81 NEAIR President 

Dawn Geronimo Terkla 
1992-93 NEAIR President 



Bill Flynn started the conference describing the 
changes in mission, vision, and structure associated 
with campus transformations into learner-centered 
institutions. In this session, former presidents of 
NEAIR will be asked to discuss past college 
transformations and how they changed institutional 
research. They will then assess the state of 
transformation on their campus — are their 
institutions embracing, ignoring, or resisting current 
forces of change? Finally, the panelists will predict 
what this means for institutional research. What is 
the role of institutional research in campus 
transformation? How will such paradigmatic shifts 
and technological advances impact our profession? 



Moderator: Michael Middaugh 

1989-90 NEAIR President Panel 



Noon - 4:00 p.m. Steering Committee Luncheon Meeting 

Reynolds 
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David W. Adams 
Director, IR 
Hunter College 
695 Dark Ave E-1701 
New York, NY 10021 
Phone: 212-650-3474 
212-650-3655 

david.adams@hunter.cuny.edu 

P 



Hershel Alexander 
Senior Rsch Analyst 
Charles Co Comm Coll 
7830 Mitchell Rd 
PO Box 910 

La Plata, MD 20646-0910 
Phone: 301-934-7621 
301-934-7679 
hershela@charles.cc.md.us 
P 



Catherine Alvord 
Senior Data Analyst 
Cornell Univ 
Instl Rsch & Ping 
440 Day Hall 
Ithaca, NY 14853-2801 
Phone: 607-255-7546 
607-255-4612 
cja2@comell.edu 
P 



Sandra L. Alexander 
Projects Coordinator 
HEDS 

Franklin & Marshall College 

PO Box 3003 

Lancaster, PA 17604-3003 

Phone: 717-399-4448 

717-399-4456 

s_alexander@fandm.edu 

F 



Ronald G. Allan 
Research Manager 
Georgetown Univ 
OSFS 

G-19Healy Box 571252 
Washington, DC 20057-1252 
Phone: 202-687-8967 
202-687-6542 

allaiu" @ gunet . georgetown . edu 
F 



Jean Anderson 
Registrar/Dir, IR 
King’s College 
133 N River St 
Wilkes-Barre, PA 18711 
Phone: 717-208-5870 
717-825-9049 
jpanders@kings.edu 
P 
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Michelle S. Appel 
Research Analyst 
Anne Arundel Comm Coll 
101 College Pkwy 
Arnold, MD 21012 
Phone: 410-541-2741 
410-541-2245 

msappel@mail.aacc.cc.md.us 

F 



Wayne S. Arndt 
Director, IR 
Georgian Court Coll 
900 Lakewood Ave 
Lakewood, NJ 08701-2697 
Phone: 732-364-2200 x238 
732-367-3920 
wamdt @ georgian.edu 
P 



Sister Virginia Assumpta 
Director, IR & Planning 
Immaculata College 
P.O. Box 702 

Immaculata, PA 19345-0702 

Phone: 610-647-4400 X3147 

610-251-1668 

vassump 1 @ immaculata.edu 

P 



Kelli Armstrong 
Director, IR 
UMASS-Boston 
President’s Office 
One Beacon St, 26th FI 
Boston, MA 02108 
Phone: 617-287-7125 
617-287-7044 

karmstrong @ email .umassp .edu 
P 



Marlene Amo 
Director, IR 
Erie Comm Coll 
6205 Main St 

Williamsville, NY 14221-7095 

Phone: 716-851-1431 

716-851-1429 

amo @nstaff.sunyerie.edu 

P 



Barbara Astone 

Director, IR 

LaGuardia/CUNY 

31-10 Thomson Ave Rm E506 

Long Island, NY 1 1 101 

Phone: 718-482-6130 

718-482-6136 

barbar@lagcc.cuny.edu 

F 
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Lynn Atkinson 
Sr Rsch Analyst & Enr Coord 
Binghamton Univ 
Office of Budget & IR 
Adm 308 

Binghamton, NY 13902-6000 
Phone; 607-777-2365 
607-777-6453 
latkinso@binghamton.edu 
P 



K Tracy Barnes 
Asst Dean, Curricular Rsch 
Brown Univ 
PO Box K 

Providence, RI 02912 
Phone: 401-863-2500 
401-863-7542 

katharine_bames@brown.edu 

P 



Alison Bast 
Institutional Researcher 
Northampton Comm College 
3835 Green Pond Rd 
Bthlehem, PA 18020 
Phone: 610-861-4585 
610-861-5070 
adb @ mail .mhm.cc .pa.us 
F 



Valerie J. Bacik 

Director, Freshman Acad Ctr 

Gannon Univ 

University Square 

Erie, PA 16541 

Phone: 814-871-5571 

bacik001@gannon.edu 

M 



Heather Barnes 
Info Technologist II 
Keene State College 
229 Main St 
Keene, NH 03435-1506 
Phone: 603-358-2125 
603-358-2124 
hbames@keene.edu 
P 



Karen Bauer 
Asst Director, Instl R&P 
Univ of Delaware 
325 Hullihen Hall 
Newark, DE 19716 
Phone: 302-831-2021 
302-831-8530 
kbauer@udel.edu 
P 



Kathleen Beal 
Research Statistician 
Wright State Univ 
Colonel Glenn Hwy 
Dayton, OH 45435 
Phone: 937-775-2738 
937-775-2421 
kbeal @ wright.edu 
F 
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Cheryl Beil 

Dir, Academic Plng&Assessment 
George Washington Univ 
Rice Hall 602 
2121 TSt.,NW 
Washington, DC 20052 
Phone: 202-994-6712 
202-994-6683 
cbeil@gwu.edu 
P 



Sarah Jane Bernard 
Technical Specialist 
Bates College 
2 Andrews Rd 
Lewiston, ME 04240 
Phone: 207-786-8285 
207-786-8350 
sbemard ©abacus. bates.edu 
M 



Craig S. Billie 
Assoc, for IR 
SUNY-Central Admin 
State University Plaza 
Albany, NY 12246 
Phone: 518-443-5639 
518-443-5632 
billiecs@sysadm.suny.edu 
P 



Jack Bishop 
Director, IR 
Passaic Co CC 
One College Boulevard 
Paterson, NJ 07505 
Phone: 973-684-6741 
973-684-5843 
jbishop@pccc.cc.nj.us 



Elizabeth Beraha 

Programmer/ Analyst 

Boston Univ 

2nd FI, Rm 244 

88 1 Commonwealth Ave 

Boston, MA 02215 

Phone: 617-353-9094 

617-353-7300 

erlejb@uism.bu.edu 

P 



Patricia Biddar 

Ex Dir, Assess/Plng/Rsch 

Union Co Coll 

1033 Springfield Ave 

Cranford, NJ 07016-1599 

Phone: 908-709-7509 

908-709-0827 

biddar@hawk.uc.edu 

M 



Felice D. Billups 
Director, IR 

Rhode Island School of Design 
2 College St. 

Providence, RI 02903 
Phone: 401-454-6334 
401-454-6406 
fbillups @risd.edu 
M 
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Deanne Blackwell 
Asst Dir, Academic 
George Washington Univ 
Rice Hall 602 
2121 TSt.,NW 
Washington, DC 20052 
Phone: 202-994-2103 
202-994-6683 
weberd® gwu.edu 
F 



Marilyn H. Blaustein 
Director, IR 
UMASS-Amherst 
237 Whitmore Admin Bldg 
Box 38190 

Amherst, MA 01003-8190 
Phone: 413-545-0941 
413-545-3010 
blaustein@oirp.umass.edu 
P 



Karl Boughan 
Supervisor, IR 
Prince George’s Comm Coll 
301 Largo RdK-231 
Largo, MD 20774 
Phone: 301-322-0722 
301-808-0960 
kb2@email.pg.cc.md.us 
P 



Penny Blackwood 

Research Associate 

AICUP 

101 N Front St 

Harrisburg, PA 17101-1405 

Phone: 717-232-8649 

717-233-8574 

bl ackwood @ aicup. org 

F 



Miriam Blimm 
Data Coordinator 
Villanova Univ 
Office of Pinning & IR 
800 Lancaster Ave 
Villanova, PA 19085 
Phone: 610-519-7582 
610-519-7162 
mblimm01@email.vill.edu 
M 



Becky Brodiga 
Director, IR & Analysis 
Middlebury College 
402 Old Chapel 
Middlebury, VT 05753 
Phone: 802-443-5906 
802-443-2076 
rbrodiga@middlebury.edu 
P 
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David Brodigan 
VP, Research 
George Dahne & Assoc 
17 High St 

Midlebury, VT 05753 
Phone: 802-388-2593 
802-388-2820 
dbrodiga @ together.net 
P 



Jennifer A. Brown 

Director of IR and Policy Studies 

University of Massachusetts Boston 

Office of Institutional Research and 

Planning 

100 Morrissey Boulevard 
Boston, MA 02125 
Phone: 617-287-5428 
617 256-7173 
j ennifer .browm @ umb.edu 
P 



Kay Bumiston 

Asst to Pres, Plng/Rsch 

Mercer County Comm College 

1200 Old Trenton Rd 

PO Box B 

Trenton, NJ 08690 

Phone: 609-586-4800 x3601 

609-587-4666 

kayb@mccc.edu 

F 



Robert J. Brodnick 
Director, IR/Plng 
Shippensburg Univ 
1871 Old Main Dr 
Shippensburg, PA 17257-2299 
Phone: 717-532-1154 
717-532-1273 
rjbrod@ark.ship.edu 
P 



Barbara Bullock 
Research Analyst 
Wright State Univ 
Colonel Glenn Hwy 
Dayton, OH 45435 
Phone: 937-775-3759 
937-775-2421 
bbullock@ wright.edu 
F 



Elizabeth Bums 

Asst Director, Instl&Mkt Rsch 

Hofstra Univ 

Business Dev Ctr, Rm 224 
145 Hofstra Univ 
Hempstead, NY 11549 
Phone: 516-463-6872 
516-463-3907 
inreab@hofstra.edu 
P 
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Stephanie Bushey 
Director, Instl&Mkt Rsch 
Hofstra Univ 
22 IB Business Dev Ctr 
Hempstead, NY 1 1747 
Phone: 516-463-6853 
516-463-3907 
inrseb @ hofstra. edu 
M 



Paul H. Carmichael 
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